Jeremy Grifski – Page 109 – The Renegade Coder

At The Renegade Coder, I'm looking to start a new thread of tech fun to engage many of my users. In this thread, I plan to share a bunch of my cool projects that I'm working on, and in return I will give users the opportunity to contribute to those projects. To start, I'm going to launch this new project series with a simple Bash script. It's called the Plex Organizer script, and it was a little project to help manage media on a Plex server. It works by pulling media from a folder and sorting that media into a few categorized folders. This was just a quick project to make life a bit easier when multiple users were dumping content into the same folder. Let's get right into it! The Plex Organizer Script Code The following code snippet is the main function for the Plex Organizer script—written in bash. Most of the logic is here which makes it a great starting point. # The main function from which this script runs main () { # Test if the user supplied arguments if [ -z $1 ]; then echo "No arguments applied" exit fi # Test if plex directory exist if [ ! -d $1 ]; then echo "Failed to find "$1"" exit fi # Define paths based on user input PATH_TO_PLEX=$1 PLEX_UPLOAD=$1"/Upload" PLEX_STAGING=$1"/Staging" PLEX_VIDEOS=$PLEX_STAGING"/Videos" PLEX_MUSIC=$PLEX_STAGING"/Music" PLEX_PICTURES=$PLEX_STAGING"/Pictures" # Build staging directory build_plex $PLEX_UPLOAD $PLEX_STAGING $PLEX_VIDEOS $PLEX_PICTURES $PLEX_MUSIC # Get size of directory SIZE_OF_UPLOAD=$(du -s "${PLEX_UPLOAD}" | awk '{print $1}') # Load in LastDumpSize . $2 # If the upload directory is empty, update and exit if [ $SIZE_OF_UPLOAD = 0 ]; then echo "${PLEX_UPLOAD} is empty" sed -i 's/LastDumpSize=.*/LastDumpSize=0/' $2 exit fi # If the upload directory has not changed size, exit if [ $SIZE_OF_UPLOAD != $LastDumpSize ]; then echo "${PLEX_UPLOAD} size is changing" sed -i 's/LastDumpSize=.*/LastDumpSize='"$SIZE_OF_UPLOAD"'/' $2 exit fi echo "Moving files from upload to staging" # Otherwise, lets do this! # Get the list of everything with absolute paths! LIST_OF_UPLOAD=$(du -L -a ${PLEX_UPLOAD} | awk '{$1="";print}' | sed -e 's/^[[:space:]]*//') # From the upload list, grab each category PICTURES=$(grep -e '.PNG' -e '.png' -e '.GIF' -e '.gif' <<< "$LIST_OF_UPLOAD") MUSIC=$(grep -e '.mp3' <<< "$LIST_OF_UPLOAD") VIDEOS=$(grep -e '.mp4' <<< "$LIST_OF_UPLOAD") # Tells script to split by newline IFS= # SORT! [[ ! -z "$PICTURES" ]] && mv ${PICTURES} $PLEX_PICTURES [[ ! -z "$MUSIC" ]] && mv ${MUSIC} $PLEX_MUSIC [[ ! -z "$VIDEOS" ]] && mv ${VIDEOS} $PLEX_VIDEOS } The Breakdown A quick look over the code shows that the script operates as a series of higher level functions (Don't worry if these steps seem abstract. They will be broken down a bit): Test the input pathBuild the output directoriesLoad and check the current statePerform state transition Typically, you might pull these functions out and give them a descriptive name. For instance, the testing inputs functionality could be pulled out into a verifyInputs() function. Now we can break down what exactly the script is doing. Test the Input Path At this step, the script ensures that the input path is valid and that input was given. Both are necessary to keep the program running. Otherwise, the program is killed: # Test if the user supplied arguments if [ -z $1 ]; then echo "No arguments applied" exit fi # Test if plex directory exist if [ ! -d $1 ]; then echo "Failed to find "$1"" exit fi Build the Output Directories The way this tool works is it takes in a path that a user has declared to contain media. If the directory exists, this tool creates a pair of folders in that directory: Upload and Staging. # Build staging directory build_plex $PLEX_UPLOAD $PLEX_STAGING $PLEX_VIDEOS $PLEX_PICTURES $PLEX_MUSIC The upload folder should already exist, but the tool will create it if it does not. The upload folder is the location where you would tell your users to dump their content. In this case, the upload folder contains symbolic links to a series of Plex folders that were assigned to various users. Meanwhile, the staging folder would then be where the organized content would be dumped. That is why the staging directory also nests three categorized folders: Videos, Music, and Pictures. Load and Check the Current State At this point, the logic kicks off. This script was designed to run as a cronjob on a Linux system, so some state information would need to be stored between executions. The state information was decided to be stored in an external file which could be read and written every run. This state information includes the size of the upload directory during the previous run of the script. For this implementation, the script uses a potentially dangerous command for loading the state. . $2 This command sources the file at index 2 of the script input. Sourcing a file executes that file in the context of the bash script, so dangerous files can be pulled in and executed. However, for this application the call seems fairly low risk. It works by running a variable declaration command which can be accessed by subsequent lines. This is not to say that someone could not edit the file, but for practical purposes the provided file will not kill a machine. Earlier, we mentioned that the script checks to see if the upload directory is empty. If it is, there is no point in continuing. Likewise, if the size of the upload directory has changed according to the file it sourced in the last step, then the script will log the new size and exit. Otherwise, the script knows that the size of the upload directory is greater than zero and it has not changed since the last execution. This means it is okay to start organizing the data. Perform State Transition Finally, let's take a look at something interesting: # Get the list of everything with absolute paths! LIST_OF_UPLOAD=$(du -L -a ${PLEX_UPLOAD} | awk '{$1="";print}' | sed -e 's/^[[:space:]]*//') # From the upload list, grab each category PICTURES=$(grep -e '.PNG' -e '.png' -e '.GIF' -e '.gif' <<< "$LIST_OF_UPLOAD") MUSIC=$(grep -e '.mp3' <<< "$LIST_OF_UPLOAD") VIDEOS=$(grep -e '.mp4' <<< "$LIST_OF_UPLOAD") # Tells script to split by newline IFS= # SORT! [[ ! -z "$PICTURES" ]] && mv ${PICTURES} $PLEX_PICTURES [[ ! -z "$MUSIC" ]] && mv ${MUSIC} $PLEX_MUSIC [[ ! -z "$VIDEOS" ]] && mv ${VIDEOS} $PLEX_VIDEOS At this point, the script uses a fairly complicated string of pipes to pull in a list of files with absolute paths into a variable. The files are then split up into three categories using a grep for specific extensions. By no means is the list of extensions complete, but you get the idea. The script then configures the current environment to split by newlines before moving each file to their respective locations. Of course, some of these commands are complicated, so let's break them down further. # Get the list of everything with absolute paths! LIST_OF_UPLOAD=$(du -L -a ${PLEX_UPLOAD} | awk '{$1="";print}' | sed -e 's/^[[:space:]]*//') When grabbing the entire list of files that were uploaded, the script calls this command. Here, the script runs a du command which prints disk usage. In particular, this command was used to list the absolute paths of all the files in the upload directory. The key here is that du gets all of the files recursively and with absolute paths. The results are sent to the awk command which just cleans up the output such that we only get a list of absolute paths. From there, the results are sent to sed which trims whitespace from the paths. Once the list is generated, grep is used to search the list of upload files for specific extensions: PICTURES=$(grep -e '.PNG' -e '.png' -e '.GIF' -e '.gif' <<< "$LIST_OF_UPLOAD") In the example above, grep uses a set of regular expressions to generate a list of pictures. In particular, grep runs a search for files that end with .PNG, .png, .GIF, and .gif. The syntax in this example is opposite of the last example. Instead of having the output piped to grep, the output is redirected to grep in the appropriate location. In both cases, the entire command is wrapped in parentheses to initiate a subshell. The dollar sign is then used to ensure a value is returned from the subshell. The Plex Organizer Script Limitations If you are familiar with Linux, you might be asking yourself, "why doesn't he just use find?" Well, there were some hardware limitations with this script. The platform running the Plex server is a QNAP NAS TS-251. This NAS runs some version of BusyBox which was fairly limited in the basic Linux tool set. As a result, the script was stuck with calls to sed and awk. Fork the Repo As with anything, this script could use some improvements. Feel free to run over to the my personal GitHub and fork the QNAPScripts repository to make your changes. Of course, you can also just download the release referenced by this article and start using it.