"|", in this context, means "or" (not "pipe"). That is, [+|-] means "optional plus or minus".

As an example, the "man grep" page includes [-e PATTERN | -f FILE] - which means I can use grep -e with a pattern, or grep -f with a file, but I can't use -e and -f at the same time.

Nested brackets are allowed; for example, in the man gcc page (a bit of a long one to read through, and probably only of interest if you plan to be writing software in the very near future, but it's the first example I found), I see the following (among other options): [-Dmacro[=defn]...]; that means that I can use -Dmacro if I want, and if I do so I have the option of adding a definition to the end of the command after an equals sign (but I can't do that without the -Dmacro).