Research, Udacity Class

Udacity Classes 2: Shell Workshop

After completing the Version Control with Git course from Udacity, I decided the next helpful free course would be on using the shell. It’s a quick introduction, but I’ve been using the shell in both Linux and Windows for quite a few years so I figure this will be good just to again fill in some missing gaps in my learning.

Right away, I learned that the Windows terminal is actually not equal to the Mac or Linux ones. The Mac and Linux OS shells are based on UNIX, which is compatible with many servers since most servers are Linux-based. The Windows one is ran off of the old MS Dos command line though, and is not as robust. Fortunately, the last class we installed and used Git Bash, which is what the course recommends using anyway. πŸ™‚

Visual showing why a custom terminal for Windows needs to be used, as its lineage is not from UNIX

A terminal is a UI for running things like a bash shell or other command line programs, similar to how a web browser is a GUI for web pages.

It seems like this course is going to be fairly quick, which just small bits of trips & tricks, so from here I’ll just do some bullet points of things I learned that I want to look back at this post to remember:

  • If the shell is giving unexpected results from text commands, try encasing the text in single quotes ”
    • echo hello shello!!
    • echo ‘hello shello!!’
  • echo $COLUMNS x $LINES tells me how big my terminal is
    • $ in front of a word symbolizes a variable
    • $COLUMNS and $LINES are examples of shell variables
  • ls: List the files in the directory
  • cd: Move to a different directory
    • cd .. will move you back one folder in the directory
  • a semicolon will allow you to run back to back shell commands on a single line
    • cd Desktop/folder ; cd ..
    • this command will move to a specific folder called “folder” on my desktop, and then immediately move up one folder to the “Desktop” directory
  • If you have spaces in a folder, you must encapsulate the command in single quotes ”
    • cd ‘My Pictures’
  • To show the list of all files in the folder “pictures” there are several ways to do it:
    • ls Pictures
    • cd Pictures; ls
    • ls Pictures/../Pictures

At this point, my friend Mav joined me and we started talking about the “experimental terminal” that Windows is releasing to mimic the UNIX based alternatives. That is when it dawned on me that PowerShell would be this new terminal, and I know it’s a hot thing in technology right now (according to a tech recruiter I talked to). And according to this article, it should be usable with git, so why not?

Okay, so back to bullet points list now that I’ve switched to PowerShell to get some experience in on that:

  • Important difference between PowerShell and Git Bash:
    • Git Bash uses cd /c/ to go to the C:/ directory
    • PowerShell uses cd C:/
  • pwd will show the directory path of current working directory
  • .. refers to the parent directory
  • . refers to the current directory (example: ls . is the same as ls)
  • ls ~ will list your home directory
  • ls -l will give more detail in Git Bash, HOWEVER this command will not work in PowerShell
    • PowerShell is still different than bash, and while it has some equivalents built in for bash users easing into PowerShell, the PS commands are more verbose almost in English
    • pwd is Get-Location
    • cd is Set-Location
    • ls is Get-ChildItem
    • ls -ltr is Get-ChildItem $env:USERPROFILE\Desktop | Sort-Object -Property LastWriteTime
      • 0_0 ok so PowerShell is kind of extremely verbose it seems… may be worth sticking to bash since most servers are Linux and dear God this is just so extra

Since PowerShell is a verbose beast, I decided to go back to Git Bash and I’ll leave PS for another day.

Windows PowerShell is just a lot…
  • ls -l Desktop/*.doc will search for any doc filetypes on your desktop (make sure your working directory is the parent of the Desktop)
    • this will tell you from left to right: file permissions, number of links, owner name, owner group, file size, timestamp of last modification, and file/directory name.
  • mkdir will make a new directory (folder)
  • mv frompath topath will move a file/folder to a location
    • mv Desktop/file.doc Desktop/folder/ will move just file.doc
    • mv Desktop/* Desktop/folder/ will move all files
  • Quiz Question: Let’s say I change my mind and want to move the epub files back from Documents/Books to Documents. How can I do this? My current working directory is my home directory, and Documents is inside that directory.
    • mv Documents/Books/* Documents (move all files in Books to Documents)
    • cd Documents; mv Books/*.epub . (go to Documents, move all .epub files to current directory)
    • cd Documents/Books; mv * .. (go to Books folder, move everything to parent directory Documents)

Okay, WOW, that was a lot of info. But now I know how to list my files/folders in my working directory, make a new directory, search and copy/move files. AWESOME!

  • curl -L ‘http://google.com’ will download source code from the web for google.com
  • curl -o google.html -L ‘http://google.com” will output (-o) the source code (-L) for google.com into a HTML file named google.html
  • cat filename will read the contents and output the contents (short for concatenate)
  • less filename will show you the content of the files but only show you one page of the file at a time (/ will search, b for back)
  • rm filename will delete the file(s) you want to remove
    • THIS WILL NOT PUT IT IN THE TRASH CAN, IT WILL REMOVE THE FILE COMPLETELY UNLESS YOU USE THE FLAG -i TO PROMPT YOU
  • rmdir directory will completely remove a directory
  • grep keyword filename.txt will search a text file for any instance of the keyword and output them into the terminal (remember -o filname.txt will instead save them into a file)
  • grep keyword filename.txt | less will instead send the output into the “less” program INSTEAD of outputting to the terminal. The use of “|” is called piping, where you redirect the output of one command into another
  • Example of piping
    • curl -L ‘https://tinyurl.com/zeyq9vc’ | grep fish will download the file ONLY where the search for fish is present
    • curl -L ‘https://tinyurl.com/zeyq9vc’ | grep fish | wc -l will do the above only it will pipe the output into the wordcount program (wc) and count the number of lines (-l)
    • curl -L ‘https://tinyurl.com/zeyq9vc’ | grep -c fish alternatively, you can just use the -c (count) flag on grep to count the keyword search instead of piping to wc program
    • grep: “global regular expression print,” processes text line by line and prints any lines which match a specified pattern
    • wc: “short for word count” reads either standard input or a list of files and generates one or more of the following statistics: newline count, word count, and byte count
  • Grep actually uses Regular expressions (something I have a little background in from another Udacity class where we built our own web browser)
    • Regular expressions are also known as regexps or regexes
    • Regular expressions are used in lots of other programs too, including text editors
    • Example: ^[Rr]eg.*[Ee]x.* means “any string that starts with reg and has ex in it, but upper- and lowercase R and E are both OK.

Alright, time for another break…this last chunk included downloading information from the internet, searching and remove files, PIPING (SO COOL), regular expressions, and displaying/outputting files content. So excited about the amount of information this course is churning through!!

  • Shell is a programming language, which means it has variables
    • To set a variable: numbers=’one two three’
    • To refer to a variable: $numbers
    • Shell variable: internal variables to the shell program itself
    • Environment variable: shared with programs you run from within the shell
    • Path is an environment variable so the shell knows where the program is to run
    • Path will contain a list of places the program files are stored in your system (for example, where to find the “ls” program to run it)
    • The paths will be separated by the : character, for example: /user/local/bin:/user/bin:/bin/:/usr/sbin:/sbin
    • The Path variable will start searching the directories from left to right, meaning if there are multiple copies/versions of the same program on your system it will choose the first one it comes across in the directory
    • To add a new directory to the end of your Path: PATH=$PATH:/new/dir/here
      • If you do this in the shell, it will ONLY last until you close the shell
      • To make the changes permanent you would need to add the directory to the configuration file (see below for example and instructions how to do this)
    • In Windows Bash some variables will not exist and need to be made by running commands. For example:
      • echo $LOGNAME $PWD is for non-Windows Git Bash
      • echo $(WHOAMI) $PWD will run the command WHOAMI and output its contents
  • File containing commands are called Shell scripts
  • It is very common for programmers to install new programs in the directory bin, but the PATH variable does not know to look there
    • You COULD run the command by manually typing out the directory (example: bin/magic), or you can add the bin directory to the PATH variable (example: PATH=$PATH:/Users/student/bin)
    • Mac/Windows config file is located: .bash_profile
    • Linux config .bash_profile is only ran on “login sessions”, non “login sessions” will runa file .bashrc
      • To get around this you can add to the .bash_profile file the following code to enforce it to run the .bashrc file first if it exists:
        • if [-f ~/.bashrc ] ; then
        • source ~/.bashrc
        • fi
    • Every time you start the shell it will read from the config file, including adding statements to add to the PATH variable
    • You can even add statements to show the date or other information at start up. For example, add these lines into the config file:
      • date
      • echo “Hey there!”
  • Windows users may not have the .bash_profile file (somehow I do), but here is how to make one if not:
    • cd to your home directory (to find out home directory use echo $HOME)
    • create a file using touch .bashrc
    • edit it with vim .bashrc
    • save and close the file typing :wq and hit Enter
    • restat your Git Bash shell
    • When you open your Git bash it will create a .bash_profile for you
  • Side note: in order to rename files with a “.” at the end, you will need to use the shell (Windows will not let you do this in the GUI) by using the mv command
    • example: mv udacity-terminal-config .udacity-terminal-config
  • To customize the shell you can set the PS1 or PS2 variables in the .bash_profile config file (bashrcgenerator.com will create a custom PS1 variable to set your specifications)
  • PS1=’$ ‘ can remove the words that show the directory before the “$” in the shell
  • Alias command:
    • alias will allow you to set shorthand words for other commands, for example: alias ll=’ls -la’ now you can just type ll as the command
    • to list all the aliases you have made, just run the allias command
    • If you want the alias to be there when you close the shell, simply add them to the .bash_profile config file!
Popular aliases

That last section was pretty intense, but it really showed how the shell/bash (bash stands for born-again shell, meaning a new and improved version) is actually a programming language itself! Complete with variables and the ability to create and run scripts!

And with that, we’ve completed the course! The final information provided is additional resources to further our learning: The Bash Academy, Bash Beginners Guide, Bash Programming HOWTO, Regexr β€” Learn Regular Expressions

Leave a Reply