Editing Talk:1638: Backslashes

...Maybe it's meant to search for all Game Grumps transcripts which make mention of the "[http://gamegrumps.wikia.com/wiki/Grep Grep]" gag? [[Special:Contributions/108.162.216.55|108.162.216.55]] 15:53, 3 February 2016 (UTC)

It should be noted that this also occurs in almost every programming language where "\" is the escape character. i.e.
 print("Hello")
 > Hello
 print("\"Hello\"")
 > "Hello"
 print("\\Hello\\")
 > \Hello\
Oh, and by the way, isn't this the third comic to mention "Ba'al, the Soul Eater"? Maybe we should start a category. (Others are [http://www.explainxkcd.com/wiki/index.php/1246:_Pale_Blue_Dot 1246] (title text) and [http://www.explainxkcd.com/wiki/index.php/1419:_On_the_Phone 1419].)
[[Special:Contributions/173.245.54.29|173.245.54.29]] 06:14, 3 February 2016 (UTC)
:[[:Category:Ba'al|Did that]] before seeing you comment, so yes I agree. --[[User:Kynde|Kynde]] ([[User talk:Kynde|talk]]) 09:47, 3 February 2016 (UTC)

The last entry may also be an oblique reference to the infinitely-expandable recursive acronym "GOD = GOD Over Djinn" mentioned in Richard Hofstadter's Gödel, Escher, Bach.[[User:Taibhse|Taibhse]] ([[User talk:Taibhse|talk]]) 16:42, 3 February 2016 (UTC)

;I don't think the regex is invalid

According to <tt>man grep</tt> you need to specify the <tt>-E</tt> option to use extended regex; without it unescaped parentheses are not interpreted, so they don't need to match.

My - very wild - guess is that it was the command he used to find the line with the most special characters, but I am not confident enough to edit the article (if someone can confirm?). {{unsigned ip|141.101.66.83}}

If it was supposed to do that, it doesn't work. Running it on my bash history matches no lines, and I have lots of special characters in there [[Special:Contributions/197.234.242.243|197.234.242.243]] 07:12, 3 February 2016 (UTC)

Explain it to me like I'm dumb. What is this comic going on about? I think the explanation needs more examples like that hello, above, because that's almost understandable. --[[Special:Contributions/198.41.238.231|198.41.238.231]] 07:47, 3 February 2016 (UTC)
:I agree. But I cannot help either.--[[User:Kynde|Kynde]] ([[User talk:Kynde|talk]]) 09:51, 3 February 2016 (UTC)

This is the third time Randall has mentioned Ba'al the Soul Eater xD [[User:International Space Station|International Space Station]] ([[User talk:International Space Station|talk]]) 08:26, 3 February 2016 (UTC)
:Yes, that was already mentioned a few hours before you comment, see the first comment. --[[User:Kynde|Kynde]] ([[User talk:Kynde|talk]]) 09:51, 3 February 2016 (UTC)

After passing the regex through bash, you get <nowiki>\\[[(].*\\[\])][^)\]]*$</nowiki> That is, the literal character \, followed by [ or (, followed by any number of any characters, followed by \, followed by ] or ), followed by any number of characters that aren't ) or ], until the end of the line. [[Special:Contributions/108.162.216.44|108.162.216.44]] 08:33, 3 February 2016 (UTC)
:It sounds like you know what you are talking about. Anyone who can explain it good enough for the explanation, and correct the explanation of the title text if it is wrong to say that it would not work. I have added this as the reason for incomplete. But maybe also examples are needed for people with not programming skills/knowledge. We also enjoy xkcd ;-) --[[User:Kynde|Kynde]] ([[User talk:Kynde|talk]]) 09:51, 3 February 2016 (UTC)

For fun: 
 cat ~/.bash_history | xargs -d "\n" -n 1 -I {} bash -c 'chars="$(echo "$1" | grep -o "[a-zA-Z0-9 ]" | wc -l)"; echo "$(( 100 - $(( $chars * 100 / ${#1} )) )) $1"' _ {} | sort -nrk 1 | less

Outputs your bash_history, ordered by relative gibberishness. This was copied by hand from desktop to mobile, might well have a few typos.--[[Special:Contributions/162.158.90.208|162.158.90.208]] 10:04, 3 February 2016 (UTC)

The problem in the comic is not with regexes per se but with situations when the entered text or expression passes through several interpreters, like bash -> grep/sed/awk, or program text -> external shell command. In such cases, you have to escape backslashes for each program in the sequence, and it gets worse if you have 'real' backslashes in the final text that you're processing with the utilities (Windows' file paths, for example). See https://en.wikipedia.org/wiki/Leaning_toothpick_syndrome.
Feel free to lift this to the explanation page, since I'm not good at longer and more careful explanations than this one.
Also, gotta notice that Feedly stripped paired backslashes in the title text (probably passed it through some 'interpreter' embedded in its scripts). [[User:Aasasd|Aasasd]] ([[User talk:Aasasd|talk]]) 10:13, 3 February 2016 (UTC)

:A funny comment about the MediaWiki software, which is even worse than this comic: <code><nowiki><Nikerabbit> I looked the code for rlike and didn't find where it does this. Can you point me to it? <vvv> $pattern = preg_replace( '!(\\\\\\\\\\\\\\\\)*(\\\\\\\\)?/!', '$1\\/', $pattern ); <Nikerabbit> I thought that was ascii art :)</nowiki></code> ([https://phabricator.wikimedia.org/P110$275 source]) --[[Special:Contributions/162.158.91.215|162.158.91.215]] 10:18, 3 February 2016 (UTC)

Interestingly, I first looked at this on my phone (using <strike>Chrome</strike> Feedly for Android), but the title text did not display correctly in that the backslashes didn't appear (which was a little confusing!). In Chrome on my Windows desktop, the title text appeared correctly. [[User:Jdluk|Jdluk]] ([[User talk:Jdluk|talk]]) 11:36, 3 February 2016 (UTC)

enough with the harry potter fancruft. "elder" is a [[Wiktionary:elder|perfectly good word]]. just because you came across it for the first time in harry potter means you are *typing carefully* the kind of person that likes harry potter. unless this is a ''harry potter reference'' wiki, of course. in which case i'll prepare a complete list of every word that appears both here and there and put a list on every page. oh, right, no i won't. --[[Special:Contributions/141.101.106.161|141.101.106.161]] 12:41, 3 February 2016 (UTC)

Remember that "Elder" is used in a lot of RPGs to denote high level enemies or items. I feel like that's what Randall's referring to here, more than Harry Potter or the general sense of the term "Elder."
: +1. Between the fact that harry potter (, ages, or tribes) aren't mentioned anywhere else in the text and the comic being a progressive list, I see this being the most likely explanation. Plus the metion of demons, which are easily the most* common usage of the modifier.
:: (*) or second most, after "elder gods", who are, let's face it, also demons. [[Special:Contributions/162.158.180.125|162.158.180.125]] 14:41, 3 February 2016 (UTC)
::: I'm pretty sure that "Elder backslash" is in reference to the "Elder gods" of Lovecraft. [[Special:Contributions/173.245.54.35|173.245.54.35]] 16:51, 3 February 2016 (UTC)
:::: Note also that it's called 'The Elder Wand' not as an intensifier, as in this comic and the other examples given, but because it is literally ''made from the wood of an [https://en.wikipedia.org/wiki/Sambucus_nigra Elder Tree]'' I'm pretty sure it's not an intentional reference. -Graptor [[Special:Contributions/173.245.54.23|173.245.54.23]] 19:29, 3 February 2016 (UTC)

Attempting to add to the discussion: This regex is not necessarily invalid or incomprehensible.  It looks like he was looking for a line with a regular expression or definitely some code.  You just have to work your way through the backslashes.  Although it might be invalid depending on the precise rules.  He has some unescaped closing brackets and closing parenthesis.  If these have to always be escaped then the regex is invalid.  If however you  don't have to escape a closing bracket with no opening bracket, then things are fine.  I'm not familiar enough with grep's regex parser to know how it handles that edge case.  Presuming those unescaped paren and brackets are fine, his regex searches for:

1. A backslash

2. An opening bracket

3. An opening parenthesis (this is a character set but the only character in it is an opening paren)

4. Any number of any characters

5. A backslash

6. An opening bracket

7. A closing bracket

8. A closing paren (presuming it doesn't have to be escaped when there is no opening paren)

9. A closing bracket (presuming it doesn't have to be escaped when there is no opening bracket)

10. Any number of character that are not a closing paren or closing bracket

11. The end of the line


Basically he is looking for a string that looks like:

\[(AAAAA\[])]AAAAA

Looks like a regex to me, and it looks like this regex also doesn't escape closing paren/brackets that don't have an opening paren/bracket, so I'm guessing that he knows what he is doing and his regex is fine.  Maybe he was playing regex golf?
[[User:Cmancone|Cmancone]] ([[User talk:Cmancone|talk]])cmancone

Ninjaed by Cmancone, above. I agree with that result in every respect except for the start-of-string being potentially anything, but putting my own analysis in here because it took long enough to type!
 Depth-of-backslash might depend upon depth of utility. In Perl, <nowiki>''</nowiki>-quotes (among others) treat everything within as literal whilst ""-quotes (and variations) interpolates any special characters, variables, etc that you put in it.  (Search for "Quote and Quote-like operators" in your favourite PerlDocs source.)  '\sss' is a literal backslash followed by three 's' characters , while "\sss" is the special \s escape (a whitespace) followed by two further regular characters.  You might need to define the first when you need to use it to provide a not-previously-escaped \s so that it might be escaped within another context.  ''Or'' you define it as "\\sss" (escaped-\) the first time, as equivalent to '\sss'.  But '\\sss' would be a literal that, later, could be interpreted as an escaped-\ to the input of a further context where the \s finally becomes 'match a whitespace'.
 
 '\\\sss' would be literal, whilst "\\\sss" could be equivalent to '\ ss' (literal backslash, literal space, rest of characters).  Then, instead of literal '\\sss', for some purpose, you could interpolate two escaped-backslashes "\\\\sss"... and so on.
 
 Meanwhile I ''think'', just from visual inspection, "'''\\\[[(].*\\\[\])][^)\]]*$'''" in Bash should obey the interpolation rules quite nicely.  The first two characters must be a literal backslash (from the escaped-backslash) and a literal open-square bracket (again, escaped).  The next open-square and the close-square shortly after depict a character class that contains only an open-parenthesis, and could have been written as '''\('''.
 
 The '''.*''' indicates zero-or-more (the asterix) instances of ''any'' character (the dot).  There is then a literal backslash (from the next '''\\''' duo) and a literal open-square (the '''\[''' pair) and close-square (the '''\]''' pair).  The ''')''' is literal and does not need escaping (as a parenthesis group had not yet been opened), as is the next ''']''' character.  To be sure, I would have written these two as the pair escapes '''\)\]''', but horses for courses...
 
 Then there's another character class (the next '''[''' and the final ''']''') required zero-or-more times (the asterix) to use up all the rest of the characters to the end (the ending '''$''' character).  As there was no '''^''' character (a.k.a. caret/circumflex/etc) at the start, the match isn't bothered about what unmatched characters appear before the original '''\('''.  This character class, however, starts with a '''^''' which in this context (the very first character of a character-class definition, not somewhere where an entire match-string starts) indicates negation of the following selection, so it is all characters ''but'' those specified, which is the regular close-parenthesis and (because it needs to be contained within a '''[]''' pair) the escaped close-square.
 
 So, all matching strings must start with '''\[(''', i.e. the backslash, open-square and open-paren.  They can continue with ''any'' further text, before then having a '''\[])]''', i.e. backslash, open-and-close-squares and close-paren, close-square.  After this, the match continues just as long as there are no non-closing square/classic brackets before the ending.
 
 The minimum matching literal string would be '''\[(\[])]''' with longer variants being of the form '''X\[(Y\[])]Z''' where X and Y can be replaced by anything (or be absent), and Z can be replaced by anything (or absent!) ''so long as it doesn't contain possibly relevent close-brackets!''. The latter stipulation is likely because the Y (and X) ''is'' allowed to contain these characters, and for some reason you don't want to confuse the test by finding some other '''\[])]''' segment within the X/Y-zones.  (In this context, it doesn't actually seem to matter too much.  But it might do in ways I haven't spotted or just be a hang-over from a prior permutation of the test.)
 
 The "grep -o" function is working on the output to the file being '''cat'''ed (there are alternate ways of doing this that some people might prefer), to only accept the lines in the file that match the '''X\[(Y\[])]Z''' string.  These lines would appear to be lines of out.txt (a fairly generic name that reveals little to its original purpose) that are well-formed for some other purpose.  A safety-escaped (i.e. not to be taken literally by any simple parser) '''[]'''-grouping containing a '''()'''-group (''not'' escaped, perhaps reasonably in context) containing potentially random text followed by an empty '''[]''' pair (again, safety-escaped).  Depending on the source, the empty '''[]'''-pair could mean many things, as with the other layers.  And the lines may end with any further text.
 
 The "out.txt" file might be the result of a prior Grep (string-search function) quote possibly scanning code for lines of particular importance by another pattern and dumping the results to out.txt for further perusal.  And then Randall finds the need to dig further into the first result by extracting just those already selected that all have the '''X\[(Y\[])Z]'''-ish pattern to them.
 
 But I could be wrong, and that's way too long for an official explanation.
(Perhaps just something like the penultimate paragraph, if we're not entirely mistaken?) [[Special:Contributions/162.158.152.89|162.158.152.89]] 14:14, 3 February 2016 (UTC)

The regex is supposed to be looking for:
 \\\      backslash
 [[(]     [ or (
 .*       any character (repeated 0 or more times)
 space    space
 \\\      backslash
 [[\])]   probably meant to match either [, ] or ). However, it's not correct, it instead matches the literal characters [)]
 [^)\]]*  probably meant to match any character that isn't ) or ], repeated. Instead it means one character that's not a ), and then a ] zero or more times
 $        end of string

 The first problem is that you're not supposed to escape ] in a [...], and it also has to be first in the grouping (unless negated with a ^) It should be [][)] or something similar.
 The second problem is the same. The last bit should be [^])]*$ and not [^)\]]*$. [[User:Khris|Khris]] ([[User talk:Khris|talk]]) 14:24, 3 February 2016 (UTC)


I was reading through the regex, if using grep you run into an error with an unmatched ")".  Removing this gets a string such as \[(AAAAA\[]]AAAAA$  http://regexr.com/3cng8 [[Special:Contributions/162.158.214.230|162.158.214.230]] 14:42, 3 February 2016 (UTC)



The regex relies on several special cases (*surprise*).
First: bash double-quote expansion (see [https://www.gnu.org/software/bash/manual/html_node/Double-Quotes.html#Double-Quotes]). Perhaps non-intuitively, \\\ followed by a character that \ doesn't escape is an escaped backslash followed by a literal backslash, effectively the same as \\\\ followed by that same non-escaped character.  After bash double-quote expansion, this results in:

\\[[(].*\\[\])][^)\]]*$


grep interprets this as:

# any leading non-\ characters
# literal backslash
# character class containing [ and (
# zero or more *any* characters
# another literal backslash
# yet another literal backslash, via a character class containing only a backslash.  Note this does not contain an escaped ], as it might appear at first glance.  See [http://www.gnu.org/software/grep/manual/html_node/Character-Classes-and-Bracket-Expressions.html]
# literal )
# literal ]
# character class of anything except ), \
# zero or more ]
# end of line

Matching examples:
*echo 'asdf\[asdfasdf\\)]a]]]]]]' | grep -o "\\\[[(].*\\\[\])][^)\]]*$"
*echo '\(\\)]P' | grep -o "\\\[[(].*\\\[\])][^)\]]*$"


One key thing to understand is that \ is not a special character when it's in a bracket expression - you can't escape characters in bracket expressions. So [^)\] simply means any character other then ) or \. Also, ( and ) are just regular characters unless they are escaped in basic regular expressions - extended regular expressions reverse this rule.
----
[[Special:Contributions/108.162.216.34|108.162.216.34]] 16:14, 3 February 2016 (UTC)rb

I don't know about the regular expression in the title text, but I think the explanation is incorrect in that it starts off talking about regular expressions. Escaping backslashes is an issue with strings in programming in general. [[Special:Contributions/173.245.54.46|173.245.54.46]] 17:12, 3 February 2016 (UTC)
----
I suspect that Randall may have used the regexp in the title text to *find* malformed regular expressions in a file (out.txt) that he (or someone) had previously filled with output from some error message (or collection of error messages, or at least the output of something where a regular expression had been expected to work but had not worked as expected). [[Special:Contributions/162.158.252.227|162.158.252.227]] 19:06, 3 February 2016 (UTC)

You can use metacharacters in character classes, the only metacharacters in a character class that must be escaped are the closing square bracket (]), the backslash (\), the hyphen, and the carat and hyphen (^) if they are the first listed item in the set. The closing square bracket requires escaping because including it without would signal the end of the set otherwise, which then means the backslash must also be escaped. The hyphen must be escaped because, without it, it signals a range (unless it is listed first, then it is literal without escaping). Carat when listed first because otherwise it signals a negative set.<br>
Therefore, the end of the title text regex matches a backslash followed by either ] or ), which is then followed by any number (including none) of characters so long as they are not ] nor ) which means the whole regex can match "<span style="color:#040;">\[something\] more</span>" or "<span style="color:#040;">\(something\)more</span>" or "<span style="color:#040;">\[something\) more</span>" as well as "<span style="color:#040;">\[something\]</span>". — [[Special:Contributions/162.158.255.117|162.158.255.117]] 01:16, 4 February 2016 (UTC)

I'll add that I use an ''almost identical'' regex in my mail server for matching mailing-list subject lines which often have a format of "<span style="color:#040;">[Listname] normal subject line<span>" which made it pretty recognizable to me. — [[Special:Contributions/162.158.255.117|162.158.255.117]] 01:24, 4 February 2016 (UTC)

== Example of a match ==

First, the shell will do some escaping substitution. So, in order to easily read it, let's see what grep really receives:

 $ echo "\\\[[(].*\\\[\])][^)\]]*$"
 \\[[(].*\\[\])][^)\]]*$

Let's break it out:

* <code>\\</code> matches a <code>\</code>
* <code>[[(]</code> matches either a <code>[</code> or a <code>(</code>
* <code>.*</code> matches any series of characters until the next match
* <code>\\</code> matches a <code>\</code>
* <code>[\]</code> matches a <code>\</code>
* <code>)]</code> matches <code>)]</code>
* <code>[^)\]</code> matches anything but <code>)</code> or <code>\</code>
* <code>]*</code> matches any number of <code>]</code> (including none)
* <code>$</code> matches the end of the string

So the string '''\[aaa\]\\)]a]]]]]]''' matches!