explain xkcd - User contributions [en]

1343: Manuals

2016-03-15T06:22:21Z

Daethnir: Notes that sudo can run as any user, not just root.

{{comic
| number = 1343
| date = March 17, 2014
| title = Manuals
| image = manuals.png
| titletext = The most ridiculous offender of all is the sudoers man page, which for 15 years has started with a 'quick guide' to EBNF, a system for defining the grammar of a language. 'Don't despair', it says, 'the definitions below are annotated.'
}}

==Explanation==
The chart shows the quality of tools regarding their manual:
*If you don't even need a manual to use a certain tool, that tool tends to help solve problems effectively.
*If you do need a manual, the tool will probably solve the problems but you have to understand that manual before you can use this tool effectively.
*Much less helpful are the tools where you need a manual but it doesn't exist — these tools tend to create more problems than they solve.
*But the worst tools are where the manuals start with a description of the manual itself — which implies both that the tool is very complex and the manual is very hard to understand.

The title text refers to ''sudoers'', a config file for the unix command ''{{w|sudo}}''. ''sudo'' allows users to run a program with elevated permissions, as referenced in [[149: Sandwich]]. Man pages are collections of manuals for different tools, commands, files, and functions on Unix-like systems which can be viewed with the tool ''man''. You can type <code>man man</code> in a terminal to get the manpage for the manual program. See for instance also the comic [[912: Manual Override]]

The sudoers file specifies which users have sudo access, and which commands they are allowed to run as other users (typically root). The syntax of the file is very complex, and the manpage uses the {{w|Extended Backus–Naur Form}} (or EBNF) to describe the syntax. The sudoers man page starts off with an explanation of EBNF's grammar, which they reference throughout the rest of the man page in describing the syntax of the sudoers file. The [http://linux.die.net/man/5/sudoers sudoers man page] is very long, clocking in at 1504 lines. In contrast, the [http://linux.die.net/man/1/man manpages man page] only has 566 lines. The number of lines may differ between some distributions and versions.

The title text also notes that the manual's assurance, "don't despair" because "the definitions below are annotated", fails to be reassuring, and instead merely emphasizes the length and complexity of the document to read.

==Transcript==
:[A horizontal line has four points labeled on it, with the second point from the left marked with a dashed vertical line dividing the horizontal line into two parts.]
:[An arrow labeled "Solve problems" points left from the vertical line.]
:[An arrow labeled "Create problems" points right from the vertical line.]
:[The points are labeled, from left to right, "Tools that don't need a manual", "Tools that need a manual", "Tools that need a manual but don't have one", and "Tools whose manual starts with 'how to read this manual'".]

{{comic discussion}}
[[Category:Charts]]

838: Incident

2016-03-15T06:20:25Z

Daethnir: Updates to note sudo runs a shell, does not 'log in' as root.

{{comic
| number = 838
| date = December 24, 2010
| title = Incident
| image = incident.png
| titletext = He sees you when you're sleeping, he knows when you're awake, he's copied on /var/spool/mail/root, so be good for goodness' sake.
}}

==Explanation==
This comic was posted on {{w|Christmas Eve}}. While {{w|Christmas}} is principally a {{w|Christian}} holiday celebrating the birth of {{w|Jesus}} on December 25, there are many tradition around the holiday, among them {{w|Christmas Eve#Gift_giving|a tradition}} that on {{w|Christmas Eve}} {{w|Santa Claus}} will make his round delivering gifts to good children.

[[Rob]] sits behind a Linux computer and tried to change his user account from his normal access to the access of a super user by using the command "{{w|sudo|sudo su}}". Sudo is a famous phrase in xkcd lore, made famous by comic [[149: Sandwich]]. When Rob is unable to use "sudo su" (which attempts to run a new command shell as the root user) but his account is not authorized, the system says that the incident "will be reported" (usually to the system administrator, so he can see if someone is making repeated attempts at accessing administrator privileges).

In the comic, however, sudo and the system report the incidents to {{w|Santa Claus}}, who, in Christmas lore, makes a list of who is naughty and who is nice. If you are nice then you get presents, while if you are naughty, you get a lump of coal. When sudo reports to Santa that Rob's account is not authorized, he puts Rob on the naughty list.

In the title text, which is a parody of the famous Christmas song, "Santa Claus Is Coming To Town", <code>/var/spool/mail/root</code> is the root (superuser) mailbox on a Linux system, where the incident described in the comic would commonly be reported to.

==Transcript==
:[Rob is sitting at a computer. The computer's prompt is shown.]
robm@homebox~$ sudo su
Password:
robm is not in the sudoers file. This incident will be reported.
robm@homebox~$ █
:[Megan approaches.]
:Rob: Hey — who does sudo report these "incidents" ''to''?
:Megan: You know, I've never checked.
:[Santa Claus is sitting at a desk supported by candy canes, with a red monitor. On the wall are two lists labeled 'naughty' and 'nice'. He is in the process of adding a line to the 'naughty' list.]

{{comic discussion}}
[[Category:Comics featuring Rob]]
[[Category:Comics featuring Megan]]
[[Category:Comics with color]]
[[Category:Linux]]
[[Category:Christmas]]

149: Sandwich

2016-03-15T06:17:38Z

Daethnir: Updates to note sudo runs as root, not some 'admin-like' account

{{comic
| number = 149
| date = August 28, 2006
| title = Sandwich
| image = sandwich.png
| titletext = Proper User Policy apparently means Simon Says.
}}

==Explanation==
On a {{w|UNIX}} computer system, users can be assigned to all kinds of rights, for example rights to access to certain directories and files to execute certain commands. The ''{{w|sudo}}'' command lets certain (authorized) users override these policies by executing the command (everything after the word "sudo" on the command line) as the root user. Root (sometimes called the superuser) has complete system powers, exempt from all access controls. One very common activity for UNIX administrators is to install or configure software using the UNIX ''{{w|Make (software)|make}}'' command, e.g. <code>% '''make install'''</code>. Often this command requires administrative permissions in order to complete successfully, which in practice means the "<code>make ''this''</code>" command will fail unless it is typed as "<code>sudo make ''this''</code>" instead. Forgetting to start the command with "sudo" is a fairly common and frustrating mistake for people who administer UNIX systems or their personal {{w|Linux}} computer. They then need to repeat the command with "sudo", whereupon the computer responds obediently and everything works smoothly.

[[Cueball]] is demanding a sandwich from his Cueball-like friend. Not being properly asked, the friend denies the request. Cueball then (ab)uses the sudo command on the friend, who then has no choice but to go and make the sandwich and now does so without complaint, because Cueball has all the rights. For anyone versed in installing system software with the <code>make</code> command, this exchange is intensely reminiscent of the analogous onscreen experience.

{{w|Simon Says}} is a children's game in which a leader gives various commands which must be followed if and only if ([[1033: Formal Logic|iff]]) the leader prefixes the command with "Simon says". The title text compares the way the computer will run some commands if they are preceded with "sudo" to the way Simon Says players are supposed to follow orders if (and only if) they are preceded with "Simon says".

Alternatively, the title text might merely be referring to the similarity between Cueball ordering his friend around with "sudo" to the Simon Says game leader ordering other players around. Wikipedia suggests the "Simon" in the name of the game may be the powerful lord Simon de Montfort, or a corruption of Cicero, both of whom were influential politicians of their day.

==Transcript==
:[Cueball is sitting on a couch, talking to a Cueball-like friend.]
:Cueball: Make me a sandwich.
:Friend: What? Make it yourself.
:Cueball: Sudo make me a sandwich.
:Friend: Okay.

{{comic discussion}}

[[Category:Comics featuring Cueball]]
[[Category:Multiple Cueballs]]
[[Category:Linux]]
[[Category:Food]]

1654: Universal Install Script

2016-03-14T14:36:56Z

Daethnir: Fleshes out issues w/ interactive requests from the package managers.

{{comic
| number = 1654
| date = March 11, 2016
| title = Universal Install Script
| image = universal_install_script.png
| titletext = The failures usually don't hurt anything, and if it installs several versions, it increases the chance that one of them is right. (Note: The 'yes' command and '2>/dev/null' are recommended additions.)
}}

==Explanation==

Most users of computers today are used to simple, easy installation of programs. You just download a .exe or a .pkg, double click it, and do what it says. Sometimes you don't even have to install anything at all, and it runs without any installation.

However, when things are more "homebrew", for example downloading source code, things are more complicated. Under {{w|Unix-like}} systems, which this universal install script is designed for, you may have to work with "build environments" and "makefiles", and command line tools. To make this process simpler, there exist repositories of programs which host either packages of source code and the things needed to build it or the pre-built programs. When you download the package, it automatically does most of the work of building the code into something executable if necessary and then installing it. However, there are many such repositories, such as "pip" and "brew", among others listed in the comic. If you only know the name of a program or package, you may not know in which repository(ies) it resides.

The script provided in the comic attempts to fix this problem, by giving a "universal install script", which contains a lot of common install commands used in various Unix-like systems. In between each of the install commands in the script is the & character, which in POSIX-compatible shells (including {{w|Bash (Unix shell)|Bash}}, a popular shell scripting language) means it should continue to run the next command without waiting for the first command to finish, also known as "running in the background". This has the effect of running all the install commands simultaneously; all output and error text provided by the will be mixed together as they are all displaying on the screen around the same time.

The script accepts the name of a program when you run it as an argument. This value is then referenced as "$1" (argument number 1). Everywhere the script says "$1", it substitutes in the name of the package you gave it. The end result is the name being tried against a large number of software repositories and package managers, and hopefully, at least one of them will be appropriate and the program will be successfully installed. Near the end, it even tries changing the current working directory to that which is assumed to hold the package to be installed, and then runs several commands which build the program from source code.

All in all, this script would probably work; it runs many standard popular repository programs and package managers, and runs the nearly-universal commands needed to build a program. Most of the commands would simply give an error and exit, but hopefully the correct one will proceed with the install.

One of the more subtle jokes in the comic is the inclusion of <code>apt-get</code> and <code>sudo apt-get</code> in the same script. Good unix practice dictates never logging in as root; instead you stay logged in as your normal user, and run system admin accounts via <code>sudo program name</code>. This prevents accidental errors and enables logging of all sensitive commands. A side effect of this, however, is that an administrator may forget to prefix her command with <code>sudo</code>, and re-running it properly the second time. This is a common joke in the Linux community, an example of which can be found at [https://twitter.com/liamosaur/status/506975850596536320 viral tweet] which shows a humerous workaround for the issue.

Since Randall's script does not use sudo for any but the <code>apt-get</code> command, there are two possibilities: the script itself was run via the root user or via sudo (in which case the <code>sudo apt-get</code> is not needed, or the script was run as a normal user, in which case all of the commands will fail (due to lacking necessary permissions) with the possible exception of the <code>sudo apt-get</code> one.

Sudo has also been used both by [[Randall]] in [[149: Sandwich]] and by Jason Fox to force Randall to let him appear on xkcd with [[824: Guest Week: Bill Amend (FoxTrot)]].

The tool <code>curl</code> downloads files from the network (e.g., the Internet). For example <code>curl http://xkcd.com/</code> downloads and displays the xkcd HTML source. The pipe <code>|</code> in the script attaches the output of the command before the pipe to the input of the command after the pipe, thus running whatever commands exist in the web content. Although this "curl|sh" pattern is a common practice for conveniently installing software, it is considered extremely unwise; you are running untrusted code without validation, there may be a MITM who modifies the code you receive, or the remote system could have been hijacked and the code made malicious. Most local package managers (e.g. <code>apt</code>, <code>yum</code>) offer digitally-signed packages that thwart this problem. You can find many examples of software providers suggesting a <code>curl|sh</code> solution at [https://curlpipesh.tumblr.com/ curlpipesh]

There appears to be a bug with the & at the end of the "git clone" line; since a git repository typically contains program source code, not executables, it may have been intended to retrieve the source code with git and then compile and install the program in the next line. In this case, the single & should be replaced with &&, an operator that will run the second command only if the first one has completed successfully. This plays into a second bug on the "configure" line, where the placement of the & means that only the "make install" command will be run asynchronously after the "configure" and "make" steps have finished in sequence. To make success as likely as possible, the two lines should be like this:

git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" && (cd "$1"; ./configure; make; make install) &

Since all commands are running in the background, any command that requires user input will stop and wait until brought to the foreground. A common request would be for a database password, or if it is allowed to restart services for the installation. This could lead to packages being only partly installed or configured. (See more about using "yes" below.)

The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. Since all the commands come from different operating systems, versions, or distributions, it is not very likely that more than one will work (with the exception of pip/easy_install and the two forms of apt-get) or even exist on the same system. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running, by making them read input from another program that writes a (nearly) endless stream of "y"s, could simplify things further. (This would not work for any curses-based menus, or to answer any more complicated questions.) Adding <code>2>/dev/null</code> to a command redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it, meaning errors (the package not existing) will not be sent to the screen.

==Transcript==
:[In the panel is a shell script which, unusual for xkcd, uses only lower case. At the top the title of the program is inlaid in the frame, which has been broken here.]
:<big>Install.sh</big>

:<nowiki>#!/bin/bash</nowiki>

:pip install "$1" &
:easy_install "$1" &
:brew install "$1" &
:npm install "$1" &
:yum install "$1" & dnf install "$1" &
:docker run "$1" &
:pkg install "$1" &
:apt-get install "$1" &
:sudo apt-get install "$1" &
:steamcmd +app_update "$1" validate &
:git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" &
:cd "$1";./configure;make;make install &
:curl "$1" | bash &

==Trivia==
*pip and easy install are package managers for {{w|Python (programming language)|Python}}
*brew is the successor/replacement for {{w|MacPorts}} and a third-party package manager for OS X
*{{w|npm (software)|npm}} is the node package manager that maintains node.js packages
*{{w|Yellowdog Updater, Modified|yum}} is the package management tool for {{w|Red Hat Enterprise Linux}} and some derivatives
*{{w|DNF (software)|dnf}} is the package management tool for {{w|Fedora (operating system)|Fedora}} since version 22
*docker run is a {{w|Docker (software)|Docker}} command that runs a given container (similar to a virtual machine)
*pkg is the package management tool on {{w|Berkeley Software Distribution|BSD systems}}
*apt-get is the package management tool of {{w|Debian}} and derivatives (e.g. Ubuntu)
*steamcmd refers to {{w|Steam (software)|Steam}}, the computer game client
*git is the revision control software used for many projects and gained a lot of traction through the {{w|GitHub}} platform
*configure/make/make install refers to the standard way of compiling software from source (on Linux/Unix)
*curl is a tool for loading data via http:// (i.e. from a website), this data is then pushed to the shell interpreter (in order to install)
**Note: While this is a security nightmare, some projects (like Homebrew) still use it as the preferred or only method of installation.

{{comic discussion}}

[[Category:Programming]]

1654: Universal Install Script

2016-03-14T14:29:56Z

Daethnir: Removes redundant paragraph about & background usage

{{comic
| number = 1654
| date = March 11, 2016
| title = Universal Install Script
| image = universal_install_script.png
| titletext = The failures usually don't hurt anything, and if it installs several versions, it increases the chance that one of them is right. (Note: The 'yes' command and '2>/dev/null' are recommended additions.)
}}

==Explanation==

Most users of computers today are used to simple, easy installation of programs. You just download a .exe or a .pkg, double click it, and do what it says. Sometimes you don't even have to install anything at all, and it runs without any installation.

However, when things are more "homebrew", for example downloading source code, things are more complicated. Under {{w|Unix-like}} systems, which this universal install script is designed for, you may have to work with "build environments" and "makefiles", and command line tools. To make this process simpler, there exist repositories of programs which host either packages of source code and the things needed to build it or the pre-built programs. When you download the package, it automatically does most of the work of building the code into something executable if necessary and then installing it. However, there are many such repositories, such as "pip" and "brew", among others listed in the comic. If you only know the name of a program or package, you may not know in which repository(ies) it resides.

The script provided in the comic attempts to fix this problem, by giving a "universal install script", which contains a lot of common install commands used in various Unix-like systems. In between each of the install commands in the script is the & character, which in POSIX-compatible shells (including {{w|Bash (Unix shell)|Bash}}, a popular shell scripting language) means it should continue to run the next command without waiting for the first command to finish, also known as "running in the background". This has the effect of running all the install commands simultaneously; all output and error text provided by the will be mixed together as they are all displaying on the screen around the same time.

The script accepts the name of a program when you run it as an argument. This value is then referenced as "$1" (argument number 1). Everywhere the script says "$1", it substitutes in the name of the package you gave it. The end result is the name being tried against a large number of software repositories and package managers, and hopefully, at least one of them will be appropriate and the program will be successfully installed. Near the end, it even tries changing the current working directory to that which is assumed to hold the package to be installed, and then runs several commands which build the program from source code.

All in all, this script would probably work; it runs many standard popular repository programs and package managers, and runs the nearly-universal commands needed to build a program. Most of the commands would simply give an error and exit, but hopefully the correct one will proceed with the install.

One of the more subtle jokes in the comic is the inclusion of <code>apt-get</code> and <code>sudo apt-get</code> in the same script. Good unix practice dictates never logging in as root; instead you stay logged in as your normal user, and run system admin accounts via <code>sudo program name</code>. This prevents accidental errors and enables logging of all sensitive commands. A side effect of this, however, is that an administrator may forget to prefix her command with <code>sudo</code>, and re-running it properly the second time. This is a common joke in the Linux community, an example of which can be found at [https://twitter.com/liamosaur/status/506975850596536320 viral tweet] which shows a humerous workaround for the issue.

Since Randall's script does not use sudo for any but the <code>apt-get</code> command, there are two possibilities: the script itself was run via the root user or via sudo (in which case the <code>sudo apt-get</code> is not needed, or the script was run as a normal user, in which case all of the commands will fail (due to lacking necessary permissions) with the possible exception of the <code>sudo apt-get</code> one.

Sudo has also been used both by [[Randall]] in [[149: Sandwich]] and by Jason Fox to force Randall to let him appear on xkcd with [[824: Guest Week: Bill Amend (FoxTrot)]].

The tool <code>curl</code> downloads files from the network (e.g., the Internet). For example <code>curl http://xkcd.com/</code> downloads and displays the xkcd HTML source. The pipe <code>|</code> in the script attaches the output of the command before the pipe to the input of the command after the pipe, thus running whatever commands exist in the web content. Although this "curl|sh" pattern is a common practice for conveniently installing software, it is considered extremely unwise; you are running untrusted code without validation, there may be a MITM who modifies the code you receive, or the remote system could have been hijacked and the code made malicious. Most local package managers (e.g. <code>apt</code>, <code>yum</code>) offer digitally-signed packages that thwart this problem. You can find many examples of software providers suggesting a <code>curl|sh</code> solution at [https://curlpipesh.tumblr.com/ curlpipesh]

There appears to be a bug with the & at the end of the "git clone" line; since a git repository typically contains program source code, not executables, it may have been intended to retrieve the source code with git and then compile and install the program in the next line. In this case, the single & should be replaced with &&, an operator that will run the second command only if the first one has completed successfully. This plays into a second bug on the "configure" line, where the placement of the & means that only the "make install" command will be run asynchronously after the "configure" and "make" steps have finished in sequence. To make success as likely as possible, the two lines should be like this:

git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" && (cd "$1"; ./configure; make; make install) &

The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. Since all the commands come from different operating systems, versions, or distributions, it is not very likely that more than one will work (with the exception of pip/easy_install and the two forms of apt-get) or even exist on the same system. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running, by making them read input from another program that writes a (nearly) endless stream of "y"s, could simplify things further. <code>2>/dev/null</code> redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it, meaning errors (the package not existing) will not be sent to the screen.

==Transcript==
:[In the panel is a shell script which, unusual for xkcd, uses only lower case. At the top the title of the program is inlaid in the frame, which has been broken here.]
:<big>Install.sh</big>

:<nowiki>#!/bin/bash</nowiki>

:pip install "$1" &
:easy_install "$1" &
:brew install "$1" &
:npm install "$1" &
:yum install "$1" & dnf install "$1" &
:docker run "$1" &
:pkg install "$1" &
:apt-get install "$1" &
:sudo apt-get install "$1" &
:steamcmd +app_update "$1" validate &
:git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" &
:cd "$1";./configure;make;make install &
:curl "$1" | bash &

==Trivia==
*pip and easy install are package managers for {{w|Python (programming language)|Python}}
*brew is the successor/replacement for {{w|MacPorts}} and a third-party package manager for OS X
*{{w|npm (software)|npm}} is the node package manager that maintains node.js packages
*{{w|Yellowdog Updater, Modified|yum}} is the package management tool for {{w|Red Hat Enterprise Linux}} and some derivatives
*{{w|DNF (software)|dnf}} is the package management tool for {{w|Fedora (operating system)|Fedora}} since version 22
*docker run is a {{w|Docker (software)|Docker}} command that runs a given container (similar to a virtual machine)
*pkg is the package management tool on {{w|Berkeley Software Distribution|BSD systems}}
*apt-get is the package management tool of {{w|Debian}} and derivatives (e.g. Ubuntu)
*steamcmd refers to {{w|Steam (software)|Steam}}, the computer game client
*git is the revision control software used for many projects and gained a lot of traction through the {{w|GitHub}} platform
*configure/make/make install refers to the standard way of compiling software from source (on Linux/Unix)
*curl is a tool for loading data via http:// (i.e. from a website), this data is then pushed to the shell interpreter (in order to install)
**Note: While this is a security nightmare, some projects (like Homebrew) still use it as the preferred or only method of installation.

{{comic discussion}}

[[Category:Programming]]

1654: Universal Install Script

2016-03-14T14:26:17Z

Daethnir: fixes /dev/null redirect interpretation.

{{comic
| number = 1654
| date = March 11, 2016
| title = Universal Install Script
| image = universal_install_script.png
| titletext = The failures usually don't hurt anything, and if it installs several versions, it increases the chance that one of them is right. (Note: The 'yes' command and '2>/dev/null' are recommended additions.)
}}

==Explanation==

Most users of computers today are used to simple, easy installation of programs. You just download a .exe or a .pkg, double click it, and do what it says. Sometimes you don't even have to install anything at all, and it runs without any installation.

However, when things are more "homebrew", for example downloading source code, things are more complicated. Under {{w|Unix-like}} systems, which this universal install script is designed for, you may have to work with "build environments" and "makefiles", and command line tools. To make this process simpler, there exist repositories of programs which host either packages of source code and the things needed to build it or the pre-built programs. When you download the package, it automatically does most of the work of building the code into something executable if necessary and then installing it. However, there are many such repositories, such as "pip" and "brew", among others listed in the comic. If you only know the name of a program or package, you may not know in which repository(ies) it resides.

The script provided in the comic attempts to fix this problem, by giving a "universal install script", which contains a lot of common install commands used in various Unix-like systems. In between each of the install commands in the script is the & character, which in POSIX-compatible shells (including {{w|Bash (Unix shell)|Bash}}, a popular shell scripting language) means it should continue to run the next command without waiting for the first command to finish, also known as "running in the background". This has the effect of running all the install commands simultaneously; all output and error text provided by the will be mixed together as they are all displaying on the screen around the same time. More about the & below.

The script accepts the name of a program when you run it as an argument. This value is then referenced as "$1" (argument number 1). Everywhere the script says "$1", it substitutes in the name of the package you gave it. The end result is the name being tried against a large number of software repositories and package managers, and hopefully, at least one of them will be appropriate and the program will be successfully installed. Near the end, it even tries changing the current working directory to that which is assumed to hold the package to be installed, and then runs several commands which build the program from source code.

All in all, this script would probably work; it runs many standard popular repository programs and package managers, and runs the nearly-universal commands needed to build a program. Most of the commands would simply give an error and exit, but hopefully the correct one will proceed with the install.

One of the more subtle jokes in the comic is the inclusion of <code>apt-get</code> and <code>sudo apt-get</code> in the same script. Good unix practice dictates never logging in as root; instead you stay logged in as your normal user, and run system admin accounts via <code>sudo program name</code>. This prevents accidental errors and enables logging of all sensitive commands. A side effect of this, however, is that an administrator may forget to prefix her command with <code>sudo</code>, and re-running it properly the second time. This is a common joke in the Linux community, an example of which can be found at [https://twitter.com/liamosaur/status/506975850596536320 viral tweet] which shows a humerous workaround for the issue.

Since Randall's script does not use sudo for any but the <code>apt-get</code> command, there are two possibilities: the script itself was run via the root user or via sudo (in which case the <code>sudo apt-get</code> is not needed, or the script was run as a normal user, in which case all of the commands will fail (due to lacking necessary permissions) with the possible exception of the <code>sudo apt-get</code> one.

Sudo has also been used both by [[Randall]] in [[149: Sandwich]] and by Jason Fox to force Randall to let him appear on xkcd with [[824: Guest Week: Bill Amend (FoxTrot)]].

The tool <code>curl</code> downloads files from the network (e.g., the Internet). For example <code>curl http://xkcd.com/</code> downloads and displays the xkcd HTML source. The pipe <code>|</code> in the script attaches the output of the command before the pipe to the input of the command after the pipe, thus running whatever commands exist in the web content. Although this "curl|sh" pattern is a common practice for conveniently installing software, it is considered extremely unwise; you are running untrusted code without validation, there may be a MITM who modifies the code you receive, or the remote system could have been hijacked and the code made malicious. Most local package managers (e.g. <code>apt</code>, <code>yum</code>) offer digitally-signed packages that thwart this problem. You can find many examples of software providers suggesting a <code>curl|sh</code> solution at [https://curlpipesh.tumblr.com/ curlpipesh]

The use of & at the end of each line causes the shell interpreter to execute the commands in parallel (asynchronously) instead of sequentially. Even if single commands fail, the rest of them will be executed. Note this is even the case for the final commands which attempt to change to the installed package, probably the only reason why this may not work completely for packages that do need compiling after being downloaded. (However, just running this script again would probably do the trick.)

There appears to be a bug with the & at the end of the "git clone" line; since a git repository typically contains program source code, not executables, it may have been intended to retrieve the source code with git and then compile and install the program in the next line. In this case, the single & should be replaced with &&, an operator that will run the second command only if the first one has completed successfully. This plays into a second bug on the "configure" line, where the placement of the & means that only the "make install" command will be run asynchronously after the "configure" and "make" steps have finished in sequence. To make success as likely as possible, the two lines should be like this:

git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" && (cd "$1"; ./configure; make; make install) &

The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. Since all the commands come from different operating systems, versions, or distributions, it is not very likely that more than one will work (with the exception of pip/easy_install and the two forms of apt-get) or even exist on the same system. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running, by making them read input from another program that writes a (nearly) endless stream of "y"s, could simplify things further. <code>2>/dev/null</code> redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it, meaning errors (the package not existing) will not be sent to the screen.

==Transcript==
:[In the panel is a shell script which, unusual for xkcd, uses only lower case. At the top the title of the program is inlaid in the frame, which has been broken here.]
:<big>Install.sh</big>

:<nowiki>#!/bin/bash</nowiki>

:pip install "$1" &
:easy_install "$1" &
:brew install "$1" &
:npm install "$1" &
:yum install "$1" & dnf install "$1" &
:docker run "$1" &
:pkg install "$1" &
:apt-get install "$1" &
:sudo apt-get install "$1" &
:steamcmd +app_update "$1" validate &
:git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" &
:cd "$1";./configure;make;make install &
:curl "$1" | bash &

==Trivia==
*pip and easy install are package managers for {{w|Python (programming language)|Python}}
*brew is the successor/replacement for {{w|MacPorts}} and a third-party package manager for OS X
*{{w|npm (software)|npm}} is the node package manager that maintains node.js packages
*{{w|Yellowdog Updater, Modified|yum}} is the package management tool for {{w|Red Hat Enterprise Linux}} and some derivatives
*{{w|DNF (software)|dnf}} is the package management tool for {{w|Fedora (operating system)|Fedora}} since version 22
*docker run is a {{w|Docker (software)|Docker}} command that runs a given container (similar to a virtual machine)
*pkg is the package management tool on {{w|Berkeley Software Distribution|BSD systems}}
*apt-get is the package management tool of {{w|Debian}} and derivatives (e.g. Ubuntu)
*steamcmd refers to {{w|Steam (software)|Steam}}, the computer game client
*git is the revision control software used for many projects and gained a lot of traction through the {{w|GitHub}} platform
*configure/make/make install refers to the standard way of compiling software from source (on Linux/Unix)
*curl is a tool for loading data via http:// (i.e. from a website), this data is then pushed to the shell interpreter (in order to install)
**Note: While this is a security nightmare, some projects (like Homebrew) still use it as the preferred or only method of installation.

{{comic discussion}}

[[Category:Programming]]

1654: Universal Install Script

2016-03-14T14:24:15Z

Daethnir: Fleshes out curl|sh usage and warnings

{{comic
| number = 1654
| date = March 11, 2016
| title = Universal Install Script
| image = universal_install_script.png
| titletext = The failures usually don't hurt anything, and if it installs several versions, it increases the chance that one of them is right. (Note: The 'yes' command and '2>/dev/null' are recommended additions.)
}}

==Explanation==

Most users of computers today are used to simple, easy installation of programs. You just download a .exe or a .pkg, double click it, and do what it says. Sometimes you don't even have to install anything at all, and it runs without any installation.

However, when things are more "homebrew", for example downloading source code, things are more complicated. Under {{w|Unix-like}} systems, which this universal install script is designed for, you may have to work with "build environments" and "makefiles", and command line tools. To make this process simpler, there exist repositories of programs which host either packages of source code and the things needed to build it or the pre-built programs. When you download the package, it automatically does most of the work of building the code into something executable if necessary and then installing it. However, there are many such repositories, such as "pip" and "brew", among others listed in the comic. If you only know the name of a program or package, you may not know in which repository(ies) it resides.

The script provided in the comic attempts to fix this problem, by giving a "universal install script", which contains a lot of common install commands used in various Unix-like systems. In between each of the install commands in the script is the & character, which in POSIX-compatible shells (including {{w|Bash (Unix shell)|Bash}}, a popular shell scripting language) means it should continue to run the next command without waiting for the first command to finish, also known as "running in the background". This has the effect of running all the install commands simultaneously; all output and error text provided by the will be mixed together as they are all displaying on the screen around the same time. More about the & below.

The script accepts the name of a program when you run it as an argument. This value is then referenced as "$1" (argument number 1). Everywhere the script says "$1", it substitutes in the name of the package you gave it. The end result is the name being tried against a large number of software repositories and package managers, and hopefully, at least one of them will be appropriate and the program will be successfully installed. Near the end, it even tries changing the current working directory to that which is assumed to hold the package to be installed, and then runs several commands which build the program from source code.

All in all, this script would probably work; it runs many standard popular repository programs and package managers, and runs the nearly-universal commands needed to build a program. Most of the commands would simply give an error and exit, but hopefully the correct one will proceed with the install.

One of the more subtle jokes in the comic is the inclusion of <code>apt-get</code> and <code>sudo apt-get</code> in the same script. Good unix practice dictates never logging in as root; instead you stay logged in as your normal user, and run system admin accounts via <code>sudo program name</code>. This prevents accidental errors and enables logging of all sensitive commands. A side effect of this, however, is that an administrator may forget to prefix her command with <code>sudo</code>, and re-running it properly the second time. This is a common joke in the Linux community, an example of which can be found at [https://twitter.com/liamosaur/status/506975850596536320 viral tweet] which shows a humerous workaround for the issue.

Since Randall's script does not use sudo for any but the <code>apt-get</code> command, there are two possibilities: the script itself was run via the root user or via sudo (in which case the <code>sudo apt-get</code> is not needed, or the script was run as a normal user, in which case all of the commands will fail (due to lacking necessary permissions) with the possible exception of the <code>sudo apt-get</code> one.

Sudo has also been used both by [[Randall]] in [[149: Sandwich]] and by Jason Fox to force Randall to let him appear on xkcd with [[824: Guest Week: Bill Amend (FoxTrot)]].

The tool <code>curl</code> downloads files from the network (e.g., the Internet). For example <code>curl http://xkcd.com/</code> downloads and displays the xkcd HTML source. The pipe <code>|</code> in the script attaches the output of the command before the pipe to the input of the command after the pipe, thus running whatever commands exist in the web content. Although this "curl|sh" pattern is a common practice for conveniently installing software, it is considered extremely unwise; you are running untrusted code without validation, there may be a MITM who modifies the code you receive, or the remote system could have been hijacked and the code made malicious. Most local package managers (e.g. <code>apt</code>, <code>yum</code>) offer digitally-signed packages that thwart this problem. You can find many examples of software providers suggesting a <code>curl|sh</code> solution at [https://curlpipesh.tumblr.com/ curlpipesh]

The use of & at the end of each line causes the shell interpreter to execute the commands in parallel (asynchronously) instead of sequentially. Even if single commands fail, the rest of them will be executed. Note this is even the case for the final commands which attempt to change to the installed package, probably the only reason why this may not work completely for packages that do need compiling after being downloaded. (However, just running this script again would probably do the trick.)

There appears to be a bug with the & at the end of the "git clone" line; since a git repository typically contains program source code, not executables, it may have been intended to retrieve the source code with git and then compile and install the program in the next line. In this case, the single & should be replaced with &&, an operator that will run the second command only if the first one has completed successfully. This plays into a second bug on the "configure" line, where the placement of the & means that only the "make install" command will be run asynchronously after the "configure" and "make" steps have finished in sequence. To make success as likely as possible, the two lines should be like this:

git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" && (cd "$1"; ./configure; make; make install) &

The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. Since all the commands come from different operating systems, versions, or distributions, it is not very likely that more than one will work (with the exception of pip/easy_install and the two forms of apt-get) or even exist on the same system. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running, by making them read input from another program that writes a (nearly) endless stream of "y"s, could simplify things further. <code>2>/dev/null</code> redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it, meaning errors (the package not existing) will be ignored.

==Transcript==
:[In the panel is a shell script which, unusual for xkcd, uses only lower case. At the top the title of the program is inlaid in the frame, which has been broken here.]
:<big>Install.sh</big>

:<nowiki>#!/bin/bash</nowiki>

:pip install "$1" &
:easy_install "$1" &
:brew install "$1" &
:npm install "$1" &
:yum install "$1" & dnf install "$1" &
:docker run "$1" &
:pkg install "$1" &
:apt-get install "$1" &
:sudo apt-get install "$1" &
:steamcmd +app_update "$1" validate &
:git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" &
:cd "$1";./configure;make;make install &
:curl "$1" | bash &

==Trivia==
*pip and easy install are package managers for {{w|Python (programming language)|Python}}
*brew is the successor/replacement for {{w|MacPorts}} and a third-party package manager for OS X
*{{w|npm (software)|npm}} is the node package manager that maintains node.js packages
*{{w|Yellowdog Updater, Modified|yum}} is the package management tool for {{w|Red Hat Enterprise Linux}} and some derivatives
*{{w|DNF (software)|dnf}} is the package management tool for {{w|Fedora (operating system)|Fedora}} since version 22
*docker run is a {{w|Docker (software)|Docker}} command that runs a given container (similar to a virtual machine)
*pkg is the package management tool on {{w|Berkeley Software Distribution|BSD systems}}
*apt-get is the package management tool of {{w|Debian}} and derivatives (e.g. Ubuntu)
*steamcmd refers to {{w|Steam (software)|Steam}}, the computer game client
*git is the revision control software used for many projects and gained a lot of traction through the {{w|GitHub}} platform
*configure/make/make install refers to the standard way of compiling software from source (on Linux/Unix)
*curl is a tool for loading data via http:// (i.e. from a website), this data is then pushed to the shell interpreter (in order to install)
**Note: While this is a security nightmare, some projects (like Homebrew) still use it as the preferred or only method of installation.

{{comic discussion}}

[[Category:Programming]]

1654: Universal Install Script

2016-03-14T14:13:43Z

Daethnir: Fixes inaccuracies about use of sudo

{{comic
| number = 1654
| date = March 11, 2016
| title = Universal Install Script
| image = universal_install_script.png
| titletext = The failures usually don't hurt anything, and if it installs several versions, it increases the chance that one of them is right. (Note: The 'yes' command and '2>/dev/null' are recommended additions.)
}}

==Explanation==

Most users of computers today are used to simple, easy installation of programs. You just download a .exe or a .pkg, double click it, and do what it says. Sometimes you don't even have to install anything at all, and it runs without any installation.

However, when things are more "homebrew", for example downloading source code, things are more complicated. Under {{w|Unix-like}} systems, which this universal install script is designed for, you may have to work with "build environments" and "makefiles", and command line tools. To make this process simpler, there exist repositories of programs which host either packages of source code and the things needed to build it or the pre-built programs. When you download the package, it automatically does most of the work of building the code into something executable if necessary and then installing it. However, there are many such repositories, such as "pip" and "brew", among others listed in the comic. If you only know the name of a program or package, you may not know in which repository(ies) it resides.

The script provided in the comic attempts to fix this problem, by giving a "universal install script", which contains a lot of common install commands used in various Unix-like systems. In between each of the install commands in the script is the & character, which in POSIX-compatible shells (including {{w|Bash (Unix shell)|Bash}}, a popular shell scripting language) means it should continue to run the next command without waiting for the first command to finish, also known as "running in the background". This has the effect of running all the install commands simultaneously; all output and error text provided by the will be mixed together as they are all displaying on the screen around the same time. More about the & below.

The script accepts the name of a program when you run it as an argument. This value is then referenced as "$1" (argument number 1). Everywhere the script says "$1", it substitutes in the name of the package you gave it. The end result is the name being tried against a large number of software repositories and package managers, and hopefully, at least one of them will be appropriate and the program will be successfully installed. Near the end, it even tries changing the current working directory to that which is assumed to hold the package to be installed, and then runs several commands which build the program from source code.

All in all, this script would probably work; it runs many standard popular repository programs and package managers, and runs the nearly-universal commands needed to build a program. Most of the commands would simply give an error and exit, but hopefully the correct one will proceed with the install.

One of the more subtle jokes in the comic is the inclusion of <code>apt-get</code> and <code>sudo apt-get</code> in the same script. Good unix practice dictates never logging in as root; instead you stay logged in as your normal user, and run system admin accounts via <code>sudo program name</code>. This prevents accidental errors and enables logging of all sensitive commands. A side effect of this, however, is that an administrator may forget to prefix her command with <code>sudo</code>, and re-running it properly the second time. This is a common joke in the Linux community, an example of which can be found at [https://twitter.com/liamosaur/status/506975850596536320 viral tweet] which shows a humerous workaround for the issue.

Since Randall's script does not use sudo for any but the <code>apt-get</code> command, there are two possibilities: the script itself was run via the root user or via sudo (in which case the <code>sudo apt-get</code> is not needed, or the script was run as a normal user, in which case all of the commands will fail (due to lacking necessary permissions) with the possible exception of the <code>sudo apt-get</code> one.

Sudo has also been used both by [[Randall]] in [[149: Sandwich]] and by Jason Fox to force Randall to let him appear on xkcd with [[824: Guest Week: Bill Amend (FoxTrot)]].

The tool <code>curl</code> downloads files from the network (e.g., the Internet). Used like <code>curl http://xkcd.com/</code> it downloads the xkcd main page and displays the HTML source code. The pipe <code>|</code> in the script attaches the output of the command before the pipe to the input of the command after the pipe. Both commands are executed concurrently. Bash is a popular shell for Unix-like operating systems. The line <code>curl "$1" | bash</code> tries to download a file from the network and to execute the download directly. 

The use of & at the end of each line causes the shell interpreter to execute the commands in parallel (asynchronously) instead of sequentially. Even if single commands fail, the rest of them will be executed. Note this is even the case for the final commands which attempt to change to the installed package, probably the only reason why this may not work completely for packages that do need compiling after being downloaded. (However, just running this script again would probably do the trick.)

There appears to be a bug with the & at the end of the "git clone" line; since a git repository typically contains program source code, not executables, it may have been intended to retrieve the source code with git and then compile and install the program in the next line. In this case, the single & should be replaced with &&, an operator that will run the second command only if the first one has completed successfully. This plays into a second bug on the "configure" line, where the placement of the & means that only the "make install" command will be run asynchronously after the "configure" and "make" steps have finished in sequence. To make success as likely as possible, the two lines should be like this:

git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" && (cd "$1"; ./configure; make; make install) &

The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. Since all the commands come from different operating systems, versions, or distributions, it is not very likely that more than one will work (with the exception of pip/easy_install and the two forms of apt-get) or even exist on the same system. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running, by making them read input from another program that writes a (nearly) endless stream of "y"s, could simplify things further. <code>2>/dev/null</code> redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it, meaning errors (the package not existing) will be ignored.

==Transcript==
:[In the panel is a shell script which, unusual for xkcd, uses only lower case. At the top the title of the program is inlaid in the frame, which has been broken here.]
:<big>Install.sh</big>

:<nowiki>#!/bin/bash</nowiki>

:pip install "$1" &
:easy_install "$1" &
:brew install "$1" &
:npm install "$1" &
:yum install "$1" & dnf install "$1" &
:docker run "$1" &
:pkg install "$1" &
:apt-get install "$1" &
:sudo apt-get install "$1" &
:steamcmd +app_update "$1" validate &
:git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" &
:cd "$1";./configure;make;make install &
:curl "$1" | bash &

==Trivia==
*pip and easy install are package managers for {{w|Python (programming language)|Python}}
*brew is the successor/replacement for {{w|MacPorts}} and a third-party package manager for OS X
*{{w|npm (software)|npm}} is the node package manager that maintains node.js packages
*{{w|Yellowdog Updater, Modified|yum}} is the package management tool for {{w|Red Hat Enterprise Linux}} and some derivatives
*{{w|DNF (software)|dnf}} is the package management tool for {{w|Fedora (operating system)|Fedora}} since version 22
*docker run is a {{w|Docker (software)|Docker}} command that runs a given container (similar to a virtual machine)
*pkg is the package management tool on {{w|Berkeley Software Distribution|BSD systems}}
*apt-get is the package management tool of {{w|Debian}} and derivatives (e.g. Ubuntu)
*steamcmd refers to {{w|Steam (software)|Steam}}, the computer game client
*git is the revision control software used for many projects and gained a lot of traction through the {{w|GitHub}} platform
*configure/make/make install refers to the standard way of compiling software from source (on Linux/Unix)
*curl is a tool for loading data via http:// (i.e. from a website), this data is then pushed to the shell interpreter (in order to install)
**Note: While this is a security nightmare, some projects (like Homebrew) still use it as the preferred or only method of installation.

{{comic discussion}}

[[Category:Programming]]