blckjck
Hello, my name is Max.
I do mobile DJ and live sound work in the Northen Illinois area.
I have been in this buisness since 2008.

I am also a licensed electrician, which comes in handy in this field at time.
 How I backup VDJ and my media

Wed 02 May 18 @ 8:37 pm

I've seen a lot of posts regarding "my hard drive crashed, I lost my data". Or, questions about "how to backup"
So, I decided to post my solution that has been tested and working for years. The initial setup is time consuming. Once setup, it backs up reliably and fast. I have tried various automatic syncing and even manual sync solutions. At some point they have either missed files, deleted needed things, put things in the wrong spot, crashed repeatedly, or asked me for more money. So, I found a route that works with the operating systems built in tools. Namely robocopy in Windows. Mac has rsync. I may take the time to add an rsync section later.

A description of what my process is:
I have a rack based computer as my main rig. I have a laptop as my "backup" rig. I have two networked hard drives as redundant backup. I had a USB drive that gets carried around in the laptop bag. The drive actually fell and does not work. I will include how I incorporate it though. If you use only external drives, I will talk about that. There is an extra folder you need to sync. It holds your library information for that drive. If you have a mix of internal and external libraries, this procedure can still be done. I can explain it if there is interest.

The rack has two hard drives, one for the OS and one for the media. The laptop has a single drive that is divided into two logical drives. I will include a link on how to separate your internal drive logically. This is not needed, but it does make this process easier. It also leaves one less thing to plug in, get damaged, get stolen.

As far as windows is concerned, both computers have the same login name. This should not be necessary if you do all of the setup as I have. There are ways to work around if you vary from this. The drive letters are C: and D:. I have moved my entire grouping of windows "Library" folders to the D: drive. This puts my Documents, Music, Videos, etc. folder on the D: drive in one main folder called Library. This is an easy procedure. Do not just drag and drop the folders. I will add a link below with instructions on migrating your library folders. A long time back I had migrated my iTunes (I used iTunes to manage my music) library from the music folder to the D: drive for various reasons. This is not necessary either. This will work wherever your media is. The script will need to be modified to fit your file paths.

For the network, each of the USB drives are shared. I can navigate the drives via the file explorer in windows. \\738a is the main root for the network share. \\738a\3TB is one drive. \\738A\S_Drive is the second. I use a Netgear Nighthawk R7000 router that has built in network drive sharing. I will not be going into how to set up network sharing in this blog. This guide will allow you to backup to an external or a shared drive the same way.

Onto the script. I run the commands from a batch file. This allows multiple commands to run in sequence without me having to enter them one at a time. You can run each command, individually, if you want. This is useful for testing each part to see if it's working. A batch file is just a text file created in a text editor like notepad (I use notepad++). It has a FILENAME.BAT extension instead of FILENAME.TXT.
Here is what my BAT file looks like for sending everything from the rack to the backups.

robocopy "D:\Libraries" "\\738a\S_Drive\Libraries" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "D:\Libraries" "\\738a\3TB\Libraries" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "d:\iTunes Media" "\\738A\S_Drive\iTunes Media" /mir /fft /r:1 /w:0 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "d:\iTunes Media" "\\738A\3TB\iTunes Media" /mir /fft /r:1 /w:0 /tee /log+:"\\738A\3TB\robolog.log"
robocopy "C:\FreeStyler" "\\738A\S_Drive\FreeStyler" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "D:\REAPER Media" "\\738a\S_Drive\REAPER Media" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "D:\PIONEER" "\\738A\3TB\PIONEER" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\3TB\robolog.log"
robocopy "D:\PIONEER" "\\738A\S_Drive\PIONEER" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
pause

rem Checking the difference between two folders.
rem ROBOCOPY “D:\Libraries” “\\738a\S_Drive\Libraries” /e /l /ns /ndl /njs /njh /fp /log+:"d:\compare.log"
rem The following two commands are used if I did not migrate my windows library folders. VDJ will create a library directory.
rem This is true for USB drives as well as logical drives.
rem robocopy "D:\VirtualDJ" "\\738a\S_Drive\VirtualDJ" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
rem robocopy "D:\VirtualDJ" "\\738a\3TB\VirtualDJ" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"

Now, I'll take apart the first line so you can see what it is doing:

robocopy "d:\iTunes Media" "\\738A\S_Drive\iTunes Media" /mir /fft /r:1 /w:0 /tee /log+:"\\738A\S_Drive\robolog.log"

We have the command itself, then the source path, then the destination path, some options, and where I want the log file written to.
For the options:
/mir Mirrors a directory tree including files and empty directories. If the file has not been modified, nothing happens.
If it is different, the source file overwrites the one in the destination. If it has been deleted from the source, the destination is deleted.

/fft Assumes FAT file times (two-second precision). I do this because the network share does not have the same timestamp setup as NTFS.

/r:1 Specifies the number of retries on failed copies. If the connection is lost, the script will keep retrying for 1 million tries, per file, without this option.

/w:0 Specifies the wait time between retries, in seconds. If there is a problem, by default, it will wait 30 seconds each file before moving on.

/tee Writes the status output to the console window, as well as to the log file. This lets you see something is happening on the screen also.

/log+: Writes the status output to the log file (appends the output to the existing log file).


robocopy "SOURCEPATH" "DESTINATIONPATH" /mir /fft /r:1 /w:0 /tee /log+:"DESTINATIONPATH\FILENAME.LOG"


Now, what is the script doing. We are looking at each of the specified source directories to see what has been changed. Then, we are mirroring those changes to the destination and keeping a log of it.

My first directory is my Libraries directory. This has my Documents, Downloads, Dropbox, Music, Pictures, and Videos folder in it. These used to be under my user name until I migrated them to the D: drive. This backs up everything from every program as long as I save them into the windows default library locations. So, if I type up a document and do a save as, it defaults to putting it into my documents folder. If I make a change to a setting in VDJ, it defaults to putting it into the VirtualDJ folder that is in the default windows Documents folder. Because it was migrated, this is now located at:

D:\Libraries\Documents\VirtualDJ


So, the first and second commands back up VDJ program stuff and more, to each of the network drives. Be aware, if you use online content, those files are cached in here as well.

robocopy "D:\Libraries" "\\738a\S_Drive\Libraries" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "D:\Libraries" "\\738a\3TB\Libraries" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"

Next, I back up my music/media. If you have your stuff in the default Library\Music folder and such, then you won't have to worry about this. I have mine in a different directory path. It's doing the same stuff as above. Mirroring the source to the destination.

robocopy "d:\iTunes Media" "\\738A\S_Drive\iTunes Media" /mir /fft /r:1 /w:0 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "d:\iTunes Media" "\\738A\3TB\iTunes Media" /mir /fft /r:1 /w:0 /tee /log+:"\\738A\3TB\robolog.log"

Some programs don't follow the Windows suggested folder defaults. So, I had to get those seperately in the next part. Again, just mirroring. In this case its all the settings and information for FreestylerDMX, the library for Rekordbox, and my recording directoris for Reaper DAW.

robocopy "C:\FreeStyler" "\\738A\S_Drive\FreeStyler" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "D:\REAPER Media" "\\738a\S_Drive\REAPER Media" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"
robocopy "D:\PIONEER" "\\738A\3TB\PIONEER" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\3TB\robolog.log"
robocopy "D:\PIONEER" "\\738A\S_Drive\PIONEER" /mir /fft /w:0 /r:1 /tee /log+:"\\738A\S_Drive\robolog.log"


The last bit are remark statements. They allow you to have comments or disable a certain command from running. In this case it is a working command for comparing a source and destination for differences without making changes to them, and then putting that into a log file.

rem Checking the difference between two folders.
rem ROBOCOPY “D:\Libraries” “\\738a\S_Drive\Libraries” /e /l /ns /ndl /njs /njh /fp /log+:"d:\compare.log"

If we were going to a USB drive, the steps are the same, we just change the destination. All you need to know is the drive letter and substitute it for the paths. Also, if you do not have the wondows library folders migrated, but your music is on the USB, VDJ creates a database folder you have to copy. FOr example, let us say that out USB drive is the letter G:

robocopy "d:\FOLDERWITHMUSIC" "g:\FOLDERWITHMUSIC" /mir /fft /r:1 /w:0 /tee /log+:"g:\robolog.log"
robocopy "d:\VirtualDJ" "g:VirtualDJ" /mir /fft /r:1 /w:0 /tee /log+:"g:\robolog.log"


Okay, I hope your still awake at this point. The first section of backing up is set. We have all this in a batch file that we run, and everything mirrors to the network or USB drive. Now, how do we get it back? From my backup laptop, I need a different batch file. A lot of what we will do is just reversing the source and destination. There is an extra thing to be concerned about. The license and settings fiels. If your backup needs different settings for VDJ than your main, you don't want to overwrite them. Same is true for the license file. VDJ keeps a seperate license for each instance of VDJ.

Here is the file from my backup laptop:

robocopy D:\Libraries\Documents\VirtualDJ D:\Libraries\Dropbox\Dropbox\MyDocVirtualDJ\LicDatTosh license.dat settings.xml database.xml /r:1 /w:0 /tee
robocopy "\\738A\S_Drive\iTunes Media" "d:\iTunes Media" /mir /fft /r:1 /w:0 /tee
robocopy "\\738A\S_Drive\PIONEER" "D:\PIONEER" /mir /fft /w:0 /r:1 /tee
robocopy "\\738A\S_Drive\FreeStyler" "C:\FreeStyler" /mir /fft /w:0 /r:1 /tee
robocopy "\\738a\S_Drive\Libraries\Documents" "D:\Libraries\Documents" /mir /fft /w:0 /r:1 /tee
robocopy "\\738a\S_Drive\Libraries\Downloads" "D:\Libraries\Downloads" /mir /fft /w:0 /r:1 /tee
robocopy "\\738a\S_Drive\Libraries\Music" "D:\Libraries\Music" /mir /fft /w:0 /r:1 /tee
robocopy "\\738a\S_Drive\Libraries\Pictures" "D:\Libraries\Pictures" /mir /fft /w:0 /r:1 /tee
pause


The first thing I am doing is copying the settings, license, and database to a different folder. In this case dropbox. I do not worry about copying them back because I open up VDJ after syncing to double check everything. In my case my rack settings work fine on my laptop. Also, when I open VDJ, I am online and logeed in. So, it gets a new license file. This keeps me from having an expired one when I get to a gig. Let's look at the command a little closer:

robocopy D:\Libraries\Documents\VirtualDJ D:\Libraries\Dropbox\Dropbox\MyDocVirtualDJ\LicDatTosh license.dat settings.xml database.xml /r:1 /w:0 /tee

The structure of the command is the same. Although, I decided not to log things on the laptop. I will probably add that later. What is different is we are not mirroring the whole directory. We are only specifying three files.

In the next 3 commands we are just reversing the source and desinations.

The last 4 command go into the libraries directory and only mirror certain subdirectories. This is because my laptop drive is not as big as my other drives. So, I have to limit what I have on it.

The last command in both of those batch files is a pause. This keeps the window open when the file is done running. I do this so I am reminded I ran the script. And, I can scroll up and see things went okay.

I keep the batch files in \Documents\VirtualDJ\Tools. I then create a shortcut to my desktop. This keeps any edits to the files synced as well.

I welcome feedback and will continue to edit this if it helps.


Above mentioned links:

Moving your libraries to another drive/partition:
[link]https://www.pcworld.com/article/3025345/windows/move-your-windows-10-libraries-to-a-separate-drive-or-partition.html[/link]

Resizing your existing drive and adding a partition for another virtual drive:
[link]https://support.microsoft.com/en-us/help/970639/microsoft-support-how-to-re-partition-without-destroying-the-data[/link]

EDIT: 201805021550 I forgot to add where I keep the batch files. I've added that to the end.