Two Approaches for Opening Multiple Electron Windows in a React/Electron Desktop Application

Recent­ly I’ve had to fig­ure out how to open mul­ti­ple Elec­tron win­dows for use in a React/Electron appli­ca­tion. This turned out to be sim­pler and eas­i­er than I’d orig­i­nal­ly thought but find­ing the sim­ple solu­tion did take some search­ing around, since I did­n’t see any good exam­ples. In fact, many of the exam­ples I came across took me in a direc­tion that ulti­mate­ly was more com­pli­cat­ed and frus­trat­ing than necessary.

There are two ways that I know of to open a new brows­er win­dow if you’re using React/Electron:

  1. Enabling nativeWin­dowOpen in your main Browser­Win­dow, using JS window.open() to open a win­dow and tying the win­dow body to your app using cre­atePor­tal from the ‘react-dom’ package.
  2. Using an Elec­tron Inter­process Com­mu­ni­ca­tion (IPC) to send a mes­sage to your elec­tron event han­dler and spawn­ing a new BrowserWindow.

My hope here is to pro­vide the exam­ple for oth­ers to use, so they don’t have to fig­ure it out from scratch as I did. Either way may work for your sit­u­a­tion but there are some things you should know about each.

Method 1: Native Browser Window

What most of the Exam­ples and Mes­sage Board Posts Say to Do

The exam­ples I saw sug­gest­ed updat­ing the main.js where your Elec­tron appli­ca­tion is ini­tial­ized, to allow native brows­er win­dows to open, using window.open() to open a stan­dard JavaScript win­dow, then using React cre­atePor­tal to link some react component(s) to the body of the window.

BrowserWindow Settings

let window = new BrowserWindow({ 
                width: 700, height: 500, 
                show: false, 
                title: 'my title', 
                webPreferences: { 
                  nodeIntegration: true, 
                  webSecurity: false, 
                  nativeWindowOpen: true 
                }
              });

Standalone Window Component

Ide­al­ly, to do the native win­dow method, you’ll want to cre­ate a com­po­nent to encap­su­late the win­dow han­dling. Per­haps some­thing like this would get you started.

import React, { useState, useEffect} from 'react';
import {createPortal } from 'react-router';
/**
 * COMPONENT:  STANDALONE WINDOW
 * PURPOSE:   DISPLAY THE CHILD CONTENT IN A SEPARATE BROWSER WINDOW
 */
function StandaloneWindow({ children }) {
  /**@type {[Window, React.Dispatch<Window>]} */
  const [windowHandle, setWindowHandle] = useState(null);
  
  // SIDE EFFECT: WHEN COMPONENT MOUNTS OPEN A WINDOW
  //              WHEN THE COMPONENT UNMOUNTS, CLOSE THE WINDOW
  useEffect(() => {
    let w = window.open('');
    copyStyles(window.document, w.document);
    setWindowHandle(w);
    return () => {
      if(windowHandle) {
        windowHandle.close();
      }
    }
  }, []);
  /**
   * FUNCTION:  COPY STYLES
   * PURPOSE:   COPY THE STYLE CONFIGURATION FROM ONE DOCUMENT TO 
   *            ANOTHER SO THE TARGET DOCUMENT WILL MATCH 
   *            THE SOURCE DOCUMENT
   * @param {Document} source
   * @param {Document} target
   */
  function copyStyles(source, target) {
    Array.from(source.querySelectorAll('link[rel="stylesheet"], style'))
    .forEach(link => {
      target.head.appendChild(link.cloneNode(true));
    });
  }
  if(windowHandle) {
    return createPortal(children, windowHandle.document.body);
  }
}

export default StandaloneWindow;

Rendering Your Component Using the StandaloneWindow Component and React Context

It is advan­ta­geous to cre­ate a React con­text and con­text provider, to hold ref­er­ences to any win­dows you cre­ate. This way one could sim­ply add the win­dows to a col­lec­tion in a high lev­el con­text and have them ren­der. It is pos­si­ble to add to that col­lec­tion from any­where beneath the con­text provider. 

If routes are used in the appli­ca­tion then the routes can be placed inside of the con­text provider you cre­ate so the win­dows can per­sist even if the user changes routes; this way the win­dows won’t go out of scope and there­fore go blank. 

A bare­bones exam­ple of a con­text and con­text provider would be some­thing like the following.

import React, { useState, useEffect } from 'react';

import StandaloneWindow from './StandaloneWindow';

// BASIC CONTEXT TO STORE THE POPUP WINDOWS AND PROVIDER A
// FUNCTION TO CREATE A NEW POPUP WINDOW AND HANDLE THE 
// BUSINESS LOGIC OF SAVING IT IN THE COLLECTION
export const AppContext = React.createContext({
  popupWindows: [],
  createPopupWindow: () => {}
});

// APP CONTEXT PROVIDER WHICH CAN BE USED TO ACCESS THE CONTEXT
export const AppContextProvider = ({ children }) => {

  function createPopupWindow(contents) {
    let newWindow = <StandaloneWindow>{contents}</StandaloneWindow>;
  setState((oldState) => (...oldState, pupupWindows: [...oldState.popupWindow, newWindow])
  }
  return (
    <AppContext.Provider value={state}>
      {children}
    </AppContext.Provider>
  ); 
}
export { AppContext, AppContextProvider };

Rendering The Popup Windows Stored in Your Context Provider

Once the infra­struc­ture to store the win­dows is in place you need to have the win­dows inside of a ren­der­ing func­tion, so the app will be able to ren­der and update the win­dows as needed.

Here’s an exam­ple of how this could be done in our app’s index.js, where all of our routes are defined.

import React, { useContext } from 'react';
import { render } from 'react-dom';
import { HashRouter as Router, Route Switch } from 'react-router-dom';
import { AppContextProvider, AppContext } from './AppContext';


function App() {
  return(
    <AppContextProvider>
      <Router>
        <Switch>
          <Route path="/LandingPage" component={LandingPage} />
        </Switch>
      </Router>
      <PopupWindows />  
    </AppContextProvider>  
  );
}
/**
 * COMPONENT: POPUP WINDOWS
 * PURPOSE:   DISPLAY ANY POPUP WINDOWS IN THE APP CONTEXT
 */
function PopupWindows() {
  const { popupWindows } = useContext(AppContext);
  return (<>{popupWindows}</>); 
}

render(<App />, document.getElementById('root'));

Generating a Popup From a Within a Component

The fol­low­ing bare­bones exam­ple will gen­er­ate a pop­up by call­ing the cre­atePop­up­Win­dow() func­tion from the App­Con­text. This will then pop­up a new win­dow and dis­play “Hel­lo World”.

import React, { useContext } from 'react';
import { Button } from 'react-bootstrap';

function MyTestComponent() {
  let { createPopupWindow } = useContext(AppContext); 
  function getPopup() {
    createPopupWindow(<h1>Hello World</h1>);
  }
  return (<Button onClick={() => getPopup()}>Click Me</Button>);
}
export default MyTestComponent;

Conclusions About the Native Browser Window / createPortal Method

The native brows­er win­dow method is com­pli­cat­ed and may cause unex­pect­ed errors. In my own work, I was attempt­ing to show C3 charts inside of one of these win­dows and I could not get the C3 charts to dis­play. I ulti­mate­ly had to aban­don this method because it was­n’t worth try­ing to fig­ure out what was going on with it.

Method 2: Using Electron's Inter-Process Communications (IPC) To Create a New Electron BrowserWindow

After aban­don­ing method 1, I found the IPC method of spawn­ing a win­dow and dis­play­ing com­po­nents to be far supe­ri­or both in terms of sim­plic­i­ty as well as reli­a­bil­i­ty. Any­thing that works in your main react/electron app should work in one of these win­dows, pro­vid­ed you ini­tial­ize every­thing as needed.

Updates to the Electron Main.js to Facilitate Multiple Windows

Your main.js may be quite dif­fer­ent from this but this exam­ple below should give an idea of how you could use the IPC func­tion­al­i­ty of Elec­tron to cre­ate new win­dows when­ev­er it receives a new mes­sage from your application.

const { app, BrowserWindow, ipcMain, Menu } = require('electron'); 
const uuid = require('uuid/v4');

/**@type {{id: String, window: BrowserWindow}[]}
let windows = [];
let dev = false;
/**
 * FUNCTION: CREATE REACT APP WINDOW
 * PURPOSE:  REUSABLE FUNCTION FOR CREATING NEW ELECTRON BrowserWindow INSTANCES
 */
function createReactAppWindow(windows, { dev, baseDir, reactRouterPath, reactRouteParams, windowTitle, windowWidth, windowHeight }) {
  let id = uuid();
  let window = new BrowserWindow({
    width: windowWidth,
    height: windowHeight,
    show: false,
    title: windowTitle,
    webPreferences: {
      nodeIntegration: true,
      webSecurity: false,
      nativeWindowOpen: false,
    }
  });
  browserWindowCollection.push({id, window});

  let indexPath;
  // DEV MODE, IF WE ARE RUNNING FROM SOURCE
  if(dev && process.argv.indexOf('--noDevServer') === -1) {
    indexPath = url.format({
      protocol: 'http:',
      host: 'localhost:8080',
      pathname: 'index.html',
      slashes: 'true'
    });
  // IF WE ARE IN PRODUCTION MODE, RUNNING A BUILT VERSION OF THE APP
  } else {
    indexPath = url.format({
      protocol: 'file:',
      pathname: path.join(baseDir, 'dist', 'index.html'),
      slashes: true
    });
  }

  // THIS IS WHERE WE CUSTOMIZE WHICH WINDOW WILL BE DISPLAYED BY 
  // USING REACT HASH ROUTER PATH / PARAMETERS
  // THIS WILL BE LOADED ON THE REACT SIDE TO RENDER THE RIGHT
  // COMPONENTS FOR THE USER
  if( reactRouterPath) {
    indexPath = indexPath.concat(`#${reactRouterPath}${reactRouteParams ? `/${reactRouteParams}` : ''}` );
  }
  window.loadURL(indexPath);
}

// A BASIC FUNCTION TO CREATE THE MAIN WINDOW FOR OUR APP WHICH
// THE USER SEES WHEN THEY OPEN THE APPLICATION
function createMainWindow() {
  createReactAppWindow(windows, { dev, baseDir: __dirname, reactRouterPath: '/SplashPage', reactRouteParams: null, windowTitle: 'my main window' })
}

// WHEN THE APP FIRST ACTIVATES OPEN THE WINDOW
app.on('activate', () => {
  if(windows.length === 0) {
    createMainWindow();
  }
});

// HERE WE HAVE AN IPC CHANNEL CALLED 'NEW_WINDOW' WHICH WE WILL 
// SEND MESSAGES TO FROM OUR APP, IN ORDER TO CREATE THE WINDOW
// FOR EACH KIND OF WINDOW WE WANT, WE NEED TO HAVE ANOTHER case HERE
// THIS WILL CALL OUR FUNCTION TO CREATE THE WINDOW WHENEVER IT 
// RECEIVES A MESSAGE
ipcMain.on('NEW_WINDOW', (event, arg) => {
  var argObject = arg[0];
  if(argObject && argObject.type) {
    switch(argObject.type) {
      case 'MY_NEW_WINDOW_1_TYPE':
        createReactAppWindow(windows, { dev, baseDir: __dirname, reactRouterPath: '/MyPath1', reactRouteParams: argObject.param, windowTitle: 'my test window' });
        break;
    }
  }
});

Updates to Your App to Trigger the New Window Functionality

Now that we have elec­tron ready to make new win­dows, we need to update our appli­ca­tion to be able to trig­ger new win­dows to be cre­at­ed. It’s prob­a­bly best to encap­su­late this func­tion­al­i­ty in a cus­tom react hook, to sep­a­rate con­cerns with­in your application.

Custom React Hook to Trigger Window Creation

Here’s an exam­ple of the cus­tom hook you could cre­ate to trig­ger a window.

import { ipcRenderer } from 'electron';

function useWindowManagement() {
  
  function OpenMyTestWindow(myID) {
    let paramObj = { type: 'MY_NEW_WINDOW_1_TYPE', param: myID };
    ipcRenderer.send('NEW_WINDOW', [paramObj]); 
  }

  return { OpenMyTestWindow };
}
Using The Custom Window Creation Hook in a Component
import React from 'react';
 import { Button } from 'react-bootstrap';
 // THE CUSTOM HOOKS FROM THE EXAMPLE ABOVE
 import {useWindowManagement } from './WindowManagement.hooks';
 // A BAREBONES TEST COMPONENT TO CALL THE CUSTOM HOOK
 function MyComponent() {
   return(<Button onClick={() => OpenMyTestWindow('1')} />);
 }
Creating React Routes to Display Our Components In the Windows

Now that the ground­work has been laid to dis­play the win­dow, we need to have some paths for react to uti­lize for choos­ing which com­po­nents to dis­play when the win­dow opens.

In this exam­ple, the default path elec­tron will send the user to will be Splash­Page while the pop­up will send the user to MyCompo­nent­ToDis­play.

import React from 'react';
 import { render } from 'react-dom';
 import { HashRouter as Router, Route Switch } from 'react-router-dom';
 import {MyComponentToDisplay} from './MyComponentToDisplay';
 import {SplashPage} from './SplashPage';

function App() {
  return(
  <Router>
    <Switch>
      <Route path="/SplashPage" component={SplashPage} />
      <Route path="/MyPath1" component={MyComponentToDisplay} />
    </Switch>
  </Router>
  );
}

Conclusions about the IPC Method of Creating New Windows in React/Electron

This method appears to have the great­est sim­plic­i­ty, though if you have com­pli­cat­ed lists of para­me­ters there could be some chal­lenges in this ver­sion of the solution.

It seems as though, in most sit­u­a­tions, one would want to cre­ate a new win­dow using an ID as a para­me­ter and this would work quite well.

Joining Collections in MongoDB Queries using $lookup

Note: This only works in Mon­goDB 3.2 or lat­er, be sure to update if you need this functionality!

In sit­u­a­tions where you have an Objec­tID in a col­lec­tion and you want it resolved by Mon­goDB dur­ing your query, you can accom­plish this with aggre­gate and lookup.

Let’s say we had two col­lec­tions: insur­ance­Claim and insuranceProvider.

Here’s a stripped down exam­ple of insuranceClaims

{
    _id: ObjectId("5849ca6b7a5385ddcb2ea422"),
    insurer: ObjectId("584998f690b755f6a9fc3750"),
    filed: false
}

Here’s a stripped down exam­ple of insurers:

{
    _id: ObjectId("584998f690b755f6a9fc3750"),
    name: 'foo'
}

If we want to have Mon­goDB resolve the insur­er when we query the claims, we can do so with the fol­low­ing query:

db.getCollection('insuranceClaims')
    .aggregate(
      [
        {
          "$lookup": {
            "from": "insurers", 
            "localField": "insurer", 
            "foreignField": "_id", 
            "as": "insurer_loaded"
          }
        }
      ]
    );

This would result in the fol­low­ing output:

{
    _id: ObjectId("5849ca6b7a5385ddcb2ea422"),
    insurer: ObjectId("584998f690b755f6a9fc3750"),
    filed: false,
    insurer_loaded: {
        _id: ObjectId("584998f690b755f6a9fc3750"),
        name: 'foo'
    }
}

Now we’ve effec­tive­ly had mon­go resolve the insur­ers for us!

If you have insur­ers in an array instead, there’s anoth­er step necessary.

{
    _id: ObjectId("5849ca6b7a5385ddcb2ea422"),
    filed: false,
    insurers: [{ObjectId("584998f690b755f6a9fc3750")}]
}

To accom­plish the same in this sit­u­a­tion we’ll use $unwind on the array.

db.getCollection('insuranceClaims')
  .aggregate(
    [
      {"$unwind": "$insurers"},
      {
        "$lookup": {
          "from": "insurers", 
          "localField": "insurers", 
          "foreignField": "_id", 
          "as": "insurer_loaded"
        }
      }
    ]
  );

This would pro­duce the fol­low­ing output:

{
    _id: ObjectId("5849ca6b7a5385ddcb2ea422"),
    insurer: ObjectId("584998f690b755f6a9fc3750"),
    filed: false,
    insurer_loaded: [{
        _id: ObjectId("584998f690b755f6a9fc3750"),
        name: 'foo'
    }]
}

Now that you’ve joined up the col­lec­tions, you prob­a­bly want to add in some fil­ters, to nar­row the list down to exact­ly what you want. To add a query into the mix sim­ply put the query into $match as fol­lows. This query will load up claims where the field filed is false.

  .aggregate(
    [
      {"$match": {"filed": false}},
      {"$unwind": "$insurers"},
      {
        "$lookup": {
          "from": "insurers", 
          "localField": "insurers", 
          "foreignField": "_id", 
          "as": "insurer_loaded"
        }
      }
    ]
  );

[amazon_link asins=‘1491954469,1785289985,1617291609,148420896X’ template=‘ProductCarousel’ store=‘openmindspace-20′ marketplace=‘US’ link_id=‘86735e82-d39e-11e6-9377–394c1b1958d6’]

Calculating a Modulo 256 Checksum in NodeJS for an Access 2 Laboratory Instrument

I’ve recent­ly had to fig­ure out how to send a check­sum with my mes­sages to an Access 2 instru­ment. I did not find very many good resources for doing so, so I’m going to share what I’ve got here. I got my first clues from the exam­ples in the Access 2 LIS doc­u­men­ta­tion. Unfor­tu­nate­ly, it appears that many of the check­sums in that doc­u­men­ta­tion are incor­rect, so to be sure I got ahold of some raw mes­sages from the device and their check­sums, to check my work.

Sev­er­al of the exam­ples from the doc­u­men­ta­tion are includ­ed, com­ment­ed out, at the top. Unfor­tu­nate­ly, I’ve found that not all of these were cor­rect in the documentation.

var msgs = [];
 
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161600' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '1'}); // should result in ascii [c1]=2 and [c2]=1
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161524' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '6'}); // should result in ascii [c1]=2 and [c2]=6
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161412' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '2'}); // should result in ascii [c1]=2 and [c2]=2
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161300' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '1', c2: 'E'}); // should result in ascii [c1]=1 and [c2]=E
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161224' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '3'}); // should result in ascii [c1]=2 and [c2]=3
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161148' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '8'}); // should result in ascii [c1]=2 and [c2]=8
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161112' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '1', c2: 'F'}); // should result in ascii [c1]=1 and [c2]=F
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028161036' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '4'}); // should result in ascii [c1]=2 and [c2]=4
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028140331' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '1', c2: 'F'}); // should result in ascii [c1]=1 and [c2]=F
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028140255' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '4'}); // should result in ascii [c1]=2 and [c2]=4
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028131152' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '0'}); // should result in ascii [c1]=2 and [c2]=0
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028131116' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '0'}); // should result in ascii [c1]=2 and [c2]=0
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028131040' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '1', c2: 'C'}); // should result in ascii [c1]=1 and [c2]=C
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028131004' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '1', c2: 'C'}); // should result in ascii [c1]=1 and [c2]=C
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161028130928' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: 'A'}); // should result in ascii [c1]=2 and [c2]=A
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161024120959' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '9'}); // should result in ascii [c1]=2 and [c2]=9
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161024120923' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '0'}); // should result in ascii [c1]=2 and [c2]=0
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161024120847' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '5'}); // should result in ascii [c1]=2 and [c2]=5
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161024120811' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '1', c2: 'C'}); // should result in ascii [c1]=1 and [c2]=C
msgs.push({str: '1H|\\^&|||ACCESS^511550|||||LIS||P|1|20161024120735' + String.fromCharCode(13)+ String.fromCharCode(3), c1: '2', c2: '1'}); // should result in ascii [c1]=2 and [c2]=1
 
for(var i = 0; i < msgs.length; i++) {
    var msg = msgs[i];
    console.log('');
    console.log(msg.str);
    var checksum = calcChecksum(msg.str);
    console.log('checksum: ' + checksum + '  expected checksum: ' + msg.c1 + msg.c2);
}
 
function calcChecksum(string) {
    var buf = new Buffer(string);
    // Calculate the modulo 256 checksum
    var sum = 0;
    for (var i = 0, l = buf.length; i < l; i++) {
 
        sum = (sum + buf[i]) % 256;
    }
 
    console.log('checksum (binary): ' + sum.toString(2));
    // Convert to a two byte uppercase hex value
    var chars = sum.toString(16).toUpperCase();
    if (chars.length == 1) chars = "0" + chars;
    return chars;
}

[amazon_link asins=‘1783287314,1491943122,1617292850,1785287524,1785881507’ template=‘ProductCarousel’ store=‘openmindspace-20′ marketplace=‘US’ link_id=‘aff56941-d2ad-11e6-adc6-31f5c1db1590’]

LINUX: Removing Files Older Than x Days

It can often be use­ful to remove files that are unnec­es­sary, such as log files, back­up files, etc, when it is not already done auto­mat­i­cal­ly. For­tu­nate­ly there is a very sim­ple com­mand to do just that.

Using the find com­mand, it is pos­si­ble to find the files in the fold­er you want to clean out and remove them. The fol­low­ing com­mand scans the fold­er /home/myuser/myfolder/ for files old­er than 30 days and then exe­cutes rm, to remove those files.

find /home/myuser/myfolder/* -mtime +30 -exec rm {} \;

If you want to be cau­tions, you can use the fol­low­ing com­mands to test it out:

To see what find pulls up, you can run this.

find /home/myuser/myfolder/* -mtime +30

If you want to make cer­tain the exec com­mand is giv­en the right para­me­ters, you can run it through ls.

find /home/myuser/myfolder/* -mtime +30 -exec ls -l {} \;

[amazon_link asins=‘1593273894,0596005954,151738043X,1430219971,B0043D2DY6’ template=‘ProductCarousel’ store=‘openmindspace-20’ marketplace=‘US’ link_id=‘2a76a978-d376-11e6-80ff-31f3f7a3df6d’]

Deleting Rows From a Table Referenced in the Query

If you do a fair amount of SQL, then every now and then, you’ll like­ly run into a sit­u­a­tion where you need to delete some­thing from a table; but need to also ref­er­ence that table in the query as well.

The trou­ble is MySQL won’t let you do this, in many cir­cum­stances. For­tu­nate­ly, there is a sim­ple workaround.

delete from tableA 
 where id in (
   select b.id 
   from tableB b 
   join tableA a 
     on a.tableb_id = b.id
   );

The above query would throw an error sim­i­lar to the following:

ERROR 1093 (HY000): You can't specify target table 'tableA' for update in FROM clause

We can skirt this lim­i­ta­tion by nest­ing the sub­s­e­lect inside of anoth­er select state­ment. Then it will work just fine.

delete from tableA where id in (
  select aId from (
    select b.id as bId from tableB b join tableA a 
    on a.tableb_id = b.id 
  ) as apt
);

You’ll get out­put to indi­cate that the query is successful.

Query OK, 183 rows affected (0.53 sec)

This saved me a bunch of time and kept me from hav­ing to rework my query com­plete­ly, so I am shar­ing it with you!

[amazon_link asins=‘059652708X,1449314287,0321833872,1449374026,1449312004’ template=‘ProductCarousel’ store=‘openmindspace-20′ marketplace=‘US’ link_id=‘7470313a-d376-11e6-91d4-75644fc73d90’]

Automatically Check RSYNC and Restart if Stopped

I occa­sion­al­ly use RSYNC to syn­chro­nize large direc­to­ries of files between servers. This is espe­cial­ly use­ful if you’re mov­ing a client from one serv­er to anoth­er and they have alot of sta­t­ic files that are always chang­ing. You can copy the files and sync them up, all with RSYNC and if your con­nec­tion gets cut off, it will start where it left off. It will also grab changes to files that have already been RSYNCd.

I ran into an issue with RSYNC recent­ly, where­in the RSYNC process was run­ning in the back­ground; but was ter­mi­nat­ing due to errors sim­i­lar to the fol­low­ing. These con­nec­tions were prob­a­bly relat­ed to the slow and unsta­ble con­nec­tion to the remote server.

rsync: writefd_unbuffered failed to write 998 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (888092 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]

Giv­en that I was trans­fer­ring files through a rel­a­tive­ly bad inter­net con­nec­tion and received this error a half dozen times over a cou­ple of days, I decid­ed the best way to han­dle it, would be to write a cron script. This cron script should check for the RSYNC process and start it if it isn’t running.

rsync_check.sh

Cus­tomize this script for your own pur­pose, to check for your RSYNC process and start it if it isn’t running.

#!/bin/bash
echo "checking for active rsync process"
COUNT=`ps ax | grep rsync | grep -v grep | grep -v rsync_check.sh | wc -l` # see how many are running
echo "there are $COUNT rsync related processes running";
if [ $COUNT -eq 0 ] 
then
	echo "no rsync processes running, restarting process"
	killall rsync  # prevent RSYNCs from piling up, if by some unforeseen reason there are already processes running
	rsync -avz -e "ssh" user@host.com:/mnt/syncdirectory/ /home/ccase/syncdirectory/ 
fi

Crontab Entry

Save the script in the appro­pri­ate cron direc­to­ry, or add it to the cron.d direc­to­ry and put a crontab entry in, to run it at the desired inter­val. This will have it run every 10 minutes.

*/10 * * * * ccase /etc/cron.d/rsync_check.sh

No More Worries

Now you can move onto oth­er things, with the knowl­edge that your RSYNC will not just fail and leave the work undone. It prob­a­bly would­n’t hurt to check on it at first and from time to time; but there’s alot less to wor­ry about!

[amazon_link asins=‘B00PKTGLWM,B0043GXMUM,1593273894,B01BNPT1EG,B00LB0E9B4’ template=‘ProductCarousel’ store=‘openmindspace-20’ marketplace=‘US’ link_id=‘b0b28615-d376-11e6-9827-b5269e98ea0d’]

Mounting CIFS Shares At the LINUX Command Line or in /etc/fstab

Lin­ux makes it rel­a­tive­ly easy to mount shared dri­ves either man­u­al­ly, at the com­mand line, or auto­mat­i­cal­ly, by con­fig­ur­ing an entry in /etc/fstab. Here is the basic syn­tax of our mount command.

[ccase@midas ~]$ sudo mount -t cifs  -o username=<share username>,password=<share password>,<additional options> //<name or ip of server>/<share name> <folder to mount to>

Here is an exam­ple of mount­ing our CIFS share to a fold­er named myshare. We are using the option ro to mount the share read only.

[ccase@midas ~]$ sudo mount -t cifs  -o username=admin,password=secret,ro //192.168.1.200/myshare myshare

If we want to make this auto­mat­ic, it can eas­i­ly be con­fig­ured in /etc/fstab/ to mount after the net­work comes up. Here is the basic syn­tax you would use in /etc/fstab/

//<name or ip of server>/<share name> <folder to mount to> cifs  username=<share username>,password=<share password>,_netdev,<additional options>   0 0

Here is an exam­ple of mount­ing our CIFS share auto­mat­i­cal­ly to /mnt/myshare/. We are using the option _netdev, to tell it to attempt the mount only after the net­work has come up and ro, to mount the share read only.

//192.168.1.200/myshare /mnt/myshare cifs  username=admin,password=secret,_netdev,ro   0 0

Getting the Last Modification Timestamp of a File with Stat

If we want to get just the date mod­i­fied, for a file, in a for­mat of our choos­ing. This can be done with a util­i­ty called stat.

The syn­tax is as follows:

stat -f <format> -t "<timestamp format>" <path to file>

In this exam­ple, we are print­ing just the date cre­at­ed in the for­mat YYYYMMDD_HHMMSS.

stat -f "%Sm" -t "%Y%m%d_%H%M%S" filename.txt

We are using the -f “%Sm flag to spec­i­fy that we want to print out only the date mod­i­fied. The -t “%Y%m%d_%H%M%S” sets the date format.

In my exam­ple, the out­put was:

20121130_180221

This trans­lates to Novem­ber 30, 2012 at 18:02:21.

Using the Linux Command Line to Find and Copy A Large Number of Files from a Large Archive, Preserving Metadata

One of my recent chal­lenges is to go through an archive on a NAS and find all of the .xlsx files, then copy them; pre­serv­ing as much of the file meta­da­ta (date cre­at­ed, fold­er tree, etc) as pos­si­ble, to a spec­i­fied fold­er.  After this copy, they will be gone through with anoth­er script, to rename the files, using the meta­da­ta, where they will then be processed by an appli­ca­tion, which uti­lizes the name of the file in its process.

The part I want to share here, is find­ing the files and copy­ing them to a fold­er, with meta­da­ta pre­served.  This is where the pow­er of the find util­i­ty comes in handy.

Since this is a huge archive, I want to first pro­duce a list of the files, that way I will be able to break this up into two steps. This will pro­duce a list and write it into a text file.  I am first going to run a find com­mand on the vol­ume I have mount­ed called data in my Vol­umes folder.

find /Volumes/data/archive/2012 -name '*.xlsx' > ~/archive/2012_files.txt

Now that the list is saved into a text file, I want to copy the files in the list, pre­serv­ing the file meta­da­ta and path infor­ma­tion, to my archive fold­er.  The cpio util­i­ty accepts the paths of the files to copy from stdin, then copies them to my archive folder.

cat ~/archive/2012_files.txt | cpio -pvdm ~/archive

Explicitly Setting log4j Configuration File Location

I ran into an issue recent­ly, where an exist­ing log4j.xml con­fig­u­ra­tion file was built into a jar file I was ref­er­enc­ing and I was unable to get Java to rec­og­nize anoth­er file that I want­ed it to use instead.  For­tu­nate­ly, the solu­tion to this prob­lem is fair­ly straight­for­ward and simple.

I was run­ning a stand­alone appli­ca­tion in lin­ux, via a bash shell script; but this tech­nique can be used in oth­er ways too.  You sim­ply add a para­me­ter to the JVM call like the exam­ple below.

So the syn­tax is basically:

java -Dlog4j.configuration="file:<full path to file>" -cp <classpath settings> <package name where my main function is located>

Lets say I have a file named log4j.xml in /opt/tools/myapp/ which I want to use when my appli­ca­tion runs, instead of any exist­ing log4j.xml files.  This can be done by pass­ing a JVM flag -Dlog4j.configuration to Java.

Here is an example:

java -Dlog4j.configuration="file:/opt/tools/myapp/log4j.xml" -cp $CLASSPATH  my.standalone.mainClass;

With that change, as long as your log4j file is set up prop­er­ly, your prob­lems should be behind you.

[amazon_link asins=‘0071808558,B01LXGO1I2,1590594991,1617290068’ template=‘ProductCarousel’ store=‘openmindspace-20’ marketplace=‘US’ link_id=‘2bff41de-d2ae-11e6-b98d-b35baa63e07c’]