Snowflake copies data between the OR connection and the KCP stream,
meaning that in most cases the copy loops will only terminate once the
OR connection times out. In this case the OR connection is already
closed and so calls to CloseRead and CloseWrite will generate errors.
The difficulty here is that the whole point of turbotunnel sessions is
that they are not necessarily tied to a single WebSocket connection, nor
even a single client IP address. We use a heuristic: whenever a
WebSocket connection starts that has a new ClientID, we store a mapping
from that ClientID to the IP address attached to the WebSocket
connection in a lookup table. Later, when enough packets have arrived to
establish a turbotunnel session, we recover the ClientID associated with
the session (which kcp-go has stored in the RemoteAddr field), and look
it up in the table to get an IP address. We introduce a new data type,
clientIDMap, to store the clientID-to-IP mapping during the short time
between when a WebSocket connection starts and handleSession receives a
fully fledged KCP session.
The client opts into turbotunnel mode by sending a magic token at the
beginning of each WebSocket connection (before sending even the
ClientID). The token is just a random byte string I generated. The
server peeks at the token and, if it matches, uses turbotunnel mode.
Otherwise, it unreads the token and continues in the old
one-session-per-WebSocket mode.
Some proxies currently send ?client_ip=0.0.0.0 because of an error in
how they attempt to grep the address from the client's SDP. That's
inflating our "%d/%d connections had client_ip" logs. Instead, treat
these cases as if the IP address were absent.
https://bugs.torproject.org/33157https://bugs.torproject.org/33385
Rename websocketconn.WebSocketConn to websocketconn.Conn, and
websocketconn.NewWebSocketConn to websocketconn.New
Following the guidelines at
https://blog.golang.org/package-names#TOC_3%2e
These functions are called for their side effect of sending a PT error
message on stdout; they also return a representation of the error
message as an error object for the caller to use if it wishes. These
functions *always* return a non-nil error object; it is not something to
be logged, any more than the return value of errors.New is.
The mistaken logging was added in
https://bugs.torproject.org/31794b26c7a7a733ec9dd19faed3d42e1ec
- Error strings are no longer capitalized nor end with punctuation
- Alias import
- Remove extraneous initilisation code (No need to provide zero value
for variables, because the compiler does that anyway)
Modified regular expressions to not scrub fingerprints, but catch all
instances of IPv4 and IPv6 addresses. Expanded test cases with those
suggested by dcf.
Log scrubber now scrubs addresses even in the case where they are split
across calls to the scrubber's output io.Writer.
Added test cases to test that the writer behaves correctly across split
lines.
Added a scrubber that takes all logging output to the standard logger
and passes through a series of regular expressions to replace IP
addresses with safe strings (e.g., X.X.X.X:443).
Ensure server logs to stdout are also scrubbed
This is a port of commit e3f3054f8b74caa639a6d9be09702693af9a70e7 from
meek.
In the previous commit, we changed from separate Listen and Serve steps
to always calling ListenAndServe. However, we would really like to
immediately get feedback if any errors happen in the Listen step inside
the call, because it's much better for debugging if those errors get
reported to tor through SMETHOD-ERROR--rather than reporting success to
tor and actually logging an error only in the snowflake log. So we wait
100 ms for an error to occur before deciding that the Listen succeeded.
We don't need to apply this hack to the ACME HTTP-01 listener, because
it's a plaintext listener. Unlike in the TLS case, there isn't any
internal magic that the net library does that we have to rely on. We
just call net.ListenTCP and check for an error.
This is a port of commit cea86c937dc278ba6b2100c238b1d5206bbae2f0 from
meek. Its purpose is to remove the need to copy-paste parts of
net/http.Server.ListenAndServeTLS. Here is a copy of the commit message
from meek:
The net/http package provides ListenAndServe and ListenAndServeTLS
functions, but it doesn't provide a way to set up a listener without
also entering an infinite serve loop. This matters for
ListenAndServeTLS, which sets up a lot of magic behind the scenes for
TLS and HTTP/2 support. Formerly, we had copy-pasted code from
ListenAndServeTLS, but that code has only gotten more complicated in
upstream net/http.
The price we pay for this is that it's no longer possible for a server
bindaddr to ask to listen on port 0 (i.e., a random ephemeral port).
That's because we never get a change to find out what the listening
address is, before entering the serve loop.
What we gain is HTTP/2 support; formerly our copy-pasted code had the
side effect of disabling HTTP/2, because it was copied from an older
version and did things like
config.NextProtos = []string{"http/1.1"}
The new code calls http2.ConfigureServer first, but that's not what's
providing HTTP/2 support. HTTP/2 support happens by default. The reason
we call http2.ConfigureServer is because we need to set
TLSConfig.GetCertificate, and http2.ConfigureServer is a convenient way
to initialize TLSConfig in a way that is guaranteed to work with HTTP/2.
As with commit fcc274ac68 for the broker,
we need to start using the HTTP-01 challenge type in the Snowflake
websocket server transport plugin.
https://bugs.torproject.org/25346
Current versions of tor accept USERADDR with or without a port number,
but future versions may become more strict and require the port number.
https://bugs.torproject.org/23080
This way, we don't lose state of certificates every time the process is
restarted. There's a possibility, otherwise, that if you have to restart
the server rapidly, you might run into Let's Encrypt rate limits and be
unable to create a cert for a while.
https://godoc.org/rsc.io/letsencrypt#hdr-Persistent_Storage
