Software Carpentry: Intro to Bash and version control with Git: Git Cheatsheets for Quick Reference

Key Points

Introducing the Shell
  • A shell is a program whose primary purpose is to read commands and run other programs.

  • The shell’s main advantages are its high action-to-keystroke ratio, its support for automating repetitive tasks, and its capacity to access networked machines.

  • The shell’s main disadvantages are its primarily textual nature and how cryptic its commands and operation can be.

Navigating Files and Directories
  • The file system is responsible for managing information on the disk.

  • Information is stored in files, which are stored in directories (folders).

  • Directories can also store other directories, which forms a directory tree.

  • cd path changes the current working directory.

  • ls path prints a listing of a specific file or directory; ls on its own lists the current working directory.

  • pwd prints the user’s current working directory.

  • / on its own is the root directory of the whole file system.

  • A relative path specifies a location starting from the current location.

  • An absolute path specifies a location from the root of the file system.

  • Directory names in a path are separated with / on Unix, but \ on Windows.

  • .. means ‘the directory above the current one’; . on its own means ‘the current directory’.

Working With Files and Directories
  • cp old new copies a file.

  • mkdir path creates a new directory.

  • mv old new moves (renames) a file or directory.

  • rm path removes (deletes) a file.

  • * matches zero or more characters in a filename, so *.txt matches all files ending in .txt.

  • ? matches any single character in a filename, so ?.txt matches a.txt but not any.txt.

  • Use of the Control key may be described in many ways, including Ctrl-X, Control-X, and ^X.

  • The shell does not have a trash bin: once something is deleted, it’s really gone.

  • Most files’ names are something.extension. The extension isn’t required, and doesn’t guarantee anything, but is normally used to indicate the type of data in the file.

  • Depending on the type of work you do, you may need a more powerful text editor than Nano.

Pipes and Filters
  • cat displays the contents of its inputs.

  • head displays the first 10 lines of its input.

  • tail displays the last 10 lines of its input.

  • sort sorts its inputs.

  • wc counts lines, words, and characters in its inputs.

  • command > file redirects a command’s output to a file (overwriting any existing content).

  • command >> file appends a command’s output to a file.

  • first | second is a pipeline: the output of the first command is used as the input to the second.

  • The best way to use the shell is to use pipes to combine simple single-purpose programs (filters).

Loops
  • A for loop repeats commands once for every thing in a list.

  • Every for loop needs a variable to refer to the thing it is currently operating on.

  • Use $name to expand a variable (i.e., get its value). ${name} can also be used.

  • Do not use spaces, quotes, or wildcard characters such as ‘*’ or ‘?’ in filenames, as it complicates variable expansion.

  • Give files consistent names that are easy to match with wildcard patterns to make it easy to select them for looping.

  • Use the up-arrow key to scroll up through previous commands to edit and repeat them.

  • Use Ctrl-R to search through the previously entered commands.

  • Use history to display recent commands, and !number to repeat a command by number.

Shell Scripts
  • Save commands in files (usually called shell scripts) for re-use.

  • bash filename runs the commands saved in a file.

  • $@ refers to all of a shell script’s command-line arguments.

  • $1, $2, etc., refer to the first command-line argument, the second command-line argument, etc.

  • Place variables in quotes if the values might have spaces in them.

  • Letting users decide what files to process is more flexible and more consistent with built-in Unix commands.

Finding Things
  • find finds files with specific properties that match patterns.

  • grep selects lines in files that match patterns.

  • --help is an option supported by many bash commands, and programs that can be run from within Bash, to display more information on how to use these commands or programs.

  • man command displays the manual page for a given command.

  • $(command) inserts a command’s output in place.

Automated Version Control
  • Version control is like an unlimited ‘undo’.

  • Version control also allows many people to work in parallel.

Setting Up Git
  • Use git config with the --global option to configure a user name, email address, editor, and other preferences once per machine.

Creating a Repository
  • git init initializes a repository.

  • Git stores all of its repository data in the .git directory.

Tracking Changes
  • git status shows the status of a repository.

  • Files can be stored in a project’s working directory (which users see), the staging area (where the next commit is being built up) and the local repository (where commits are permanently recorded).

  • git add puts files in the staging area.

  • git commit saves the staged content as a new commit in the local repository.

  • Write a commit message that accurately describes your changes.

Exploring History
  • git diff displays differences between commits.

  • git checkout recovers old versions of files.

Ignoring Things
  • The .gitignore file tells Git what files to ignore.

Remotes in GitHub
  • A local Git repository can be connected to one or more remote repositories.

  • Use the HTTPS protocol to connect to remote repositories until you have learned how to set up SSH.

  • git push copies changes from a local repository to a remote repository.

  • git pull copies changes from a remote repository to a local repository.

Collaborating
  • git clone copies a remote repository to create a local repository with a remote called origin automatically set up.

Conflicts
  • Conflicts occur when two or more people change the same lines of the same file.

  • The version control system does not allow people to overwrite each other’s changes blindly, but highlights conflicts so that they can be resolved.

Open Science
  • Open scientific work is more useful and more highly cited than closed.

Licensing
  • People who incorporate General Public License (GPL’d) software into their own software must make their software also open under the GPL license; most other open licenses do not require this.

  • The Creative Commons family of licenses allow people to mix and match requirements and restrictions on attribution, creation of derivative works, further sharing, and commercialization.

  • People who are not lawyers should not try to write licenses from scratch.

Citation
  • Add a CITATION file to a repository to explain how you want your work cited.

Hosting
  • Projects can be hosted on university servers, on personal domains, or on public forges.

  • Rules regarding intellectual property and storage of sensitive information apply no matter where code and data are hosted.

Supplemental: Using Git from RStudio
  • Using RStudio’s Git integration allows you to version control a project over time.

Git Cheatsheets for Quick Reference

Glossary

changeset
A group of changes to one or more files that are or will be added to a single commit in a version control repository.
commit
To record the current state of a set of files (a changeset) in a version control repository. As a noun, the result of committing, i.e. a recorded changeset in a repository. If a commit contains changes to multiple files, all of the changes are recorded together.
conflict
A change made by one user of a version control system that is incompatible with changes made by other users. Helping users resolve conflicts is one of version control’s major tasks.
HTTP
The Hypertext Transfer Protocol used for sharing web pages and other data on the World Wide Web.
merge
(a repository): To reconcile two sets of changes to a repository.
protocol
A set of rules that define how one computer communicates with another. Common protocols on the Internet include HTTP and SSH.
remote
(of a repository) A version control repository connected to another, in such way that both can be kept in sync exchanging commits.
repository
A storage area where a version control system stores the full history of commits of a project and information about who changed what, when.
resolve
To eliminate the conflicts between two or more incompatible changes to a file or set of files being managed by a version control system.
revision
A synonym for commit.
SHA-1
SHA-1 hashes is what Git uses to compute identifiers, including for commits. To compute these, Git uses not only the actual change of a commit, but also its metadata (such as date, author, message), including the identifiers of all commits of preceding changes. This makes Git commit IDs virtually unique. I.e., the likelihood that two commits made independently, even of the same change, receive the same ID is exceedingly small.
SSH
The Secure Shell protocol used for secure communication between computers.
timestamp
A record of when a particular event occurred.
version control
A tool for managing changes to a set of files. Each set of changes creates a new commit of the files; the version control system allows users to recover old commits reliably, and helps manage conflicting changes made by different users.