The race condition occurs because concurrent goroutines are
intermixing reads and writes of `WebRTCPeer.lastReceive`.
Spotted when integrating Snowflake inside OONI in
https://github.com/ooni/probe-cli/pull/373.
This fixes a stats collection bug where we were converting client
addresses between a string and net.Addr using the clientAddr function
multiple times, resulting in an empty string for all addresses.
In VSCode, the staticcheck tool emits this warning:
> should call wg.Add(1) before starting the goroutine to
> avoid a race (SA2000)go-staticcheck
To avoid this warning, just move wg.Add outside.
Send the client poll request and response in a json-encoded format in
the HTTP request body rather than sending the data in HTTP headers. This
will pave the way for using domain-fronting alternatives for the
Snowflake rendezvous.
Make a stack of cleanup functions to run (as with defer), but clear the
stack before returning if no error occurs.
Uselessly pushing the stream.Close() cleanup just before clearing the
stack is an intentional safeguard, for in case additional operations are
added before the return in the future.
Fixes#40042.
Follow up to 160ae2d
Analysis by @dcf,
> I don't think the sync.Once around logMetrics is necessary anymore.
Its original purpose was to inhibit logging on later file handles of
metrics.log, if there were more than one opened. See 171c55a9 and #29734
(comment 2593039) "Making a singleton *Metrics variable causes problems
with how Convey does tests. It shouldn't be called more than once, but
for now I'm using sync.Once on the logging at least so it's explicit."
Commit ba4fe1a7 changed it so that metrics.log is opened in main, used
to create a *log.Logger, and that same instance of *log.Logger is passed
to both NewMetrics and NewBrokerContext. It's safe to share the same
*log.Logger across multiple BrokerContext.
Doesn't seem like it needs to exist outside of the metrics struct.
Also, the call to logMetrics is moved to the constructor. A metrics
instance is only created when a BrokerContext is created, which only
happens at startup. The sync of only doing that once is left for
documentation purposes, since it doesn't hurt, but also seems redundant.
Introduce a waitgroup and done channel to ensure that both the read and
write gorouting for turbotunnel connections terminate when the
connection is closed.
The default prometheus registry exports data that may be useful for
side-channel attacks. This removes all of the default metrics and makes
sure we are only reporting snowflake metrics from the broker.
This change adds a prometheus exporter for our existing snowflake broker
metrics. Current values for the metrics can be fetched by sending a GET
request to /prometheus.
Snowflake copies data between the OR connection and the KCP stream,
meaning that in most cases the copy loops will only terminate once the
OR connection times out. In this case the OR connection is already
closed and so calls to CloseRead and CloseWrite will generate errors.
Update our dependency on pion/sdp from v2 to v3, to match pion/webrtc
v3. This requires some changes in how we parse out addresses from ice
candidates. This will ease tor browser builds of snowflake since we are
now only relying on one version of pion/sdp instead of two different
ones.
This update required two main changes to how we use the library. First,
we had to make sure we created the datachannel on the offering peer side
before creating the offer. Second, we had to make sure we wait for the
gathering of all candidates to complete since trickle-ice is enabled by
default. See the release notes for more details:
https://github.com/pion/webrtc/wiki/Release-WebRTC@v3.0.0.
Run the snowflake collection ReconnectTimeout timer in parallel to the
negotiation with the broker. This way, if the broker takes a long time
to respond the client doesn't have to wait the full timeout to respond.
Normally all dangling goroutines are terminated when the main function
exits. However, for projects that use a patched version of snowflake as
a library, these goroutines continued running as long as the main function
had not yet terminated. This commit has all open SOCKS connections close
after receiving a shutdown signal.
Each SOCKS connection has its own set of snowflakes and broker poll
loop. Since the session manager was tied to a single set of snowflakes,
this resulted in a bug where RedialPacketConn would sometimes try to
pull snowflakes from a previously melted pool. The fix is to maintain
separate smux sessions for each SOCKS connection, tied to its own
snowflake pool.
We currently don't sort the snowflake-ips metrics:
snowflake-ips CA=1,DE=1,AR=1,NL=1,FR=1,GB=2,US=4,CH=1
To facilitate eyeballing our metrics, this patch sorts snowflake-ips by
value. If the value is identical, we sort by string, i.e.:
snowflake-ips US=4,GB=2,AR=1,CA=1,CH=1,DE=1,FR=1,NL=1
This patch fixes tpo/anti-censorship/pluggable-transports/snowflake#40011
Instead of continuously polling the broker until the client receives a
snowflake, fail back to the Connect() loop and try again to collect more
peers after ReconnectTimeout.
Rather than having standalone proxies determine their NAT type by
conducting the NAT behaviour checks in RFC 5780, use the remote probe
service instead.
The easiest way to set up the probe server behind a symmetric NAT is to
deploy it as a Docker container and alter the iptables rules for the
Docker network subnet that the container runs in.
We expect one of these at the end of just about every proxy session, as
the Conns in both directions are closed as soon as the copy loop
finishes in one direction.
Closes#40016.
This fixes a race condition in which snowflakes.End() is called while
snowflakes.Collect() is in progress resulting in a write to a closed
channel. We now wait for all in-progress collections to finish and add
an extra check before proceeding with a collection.
