VPN Connection Troubleshooting
VPN Connection Troubleshooting
Section titled “VPN Connection Troubleshooting”This guide explains how ARROW devices establish their VPN connection, which ports they need open, and how to work out exactly what a client’s network is blocking.
How the VPN connects
Section titled “How the VPN connects”ARROW uses NetBird, an open-source VPN built on WireGuard. Getting a device connected happens in four steps. The device authenticates with NetBird’s management API, the peers exchange connection details through the signal service, they try to build a direct WireGuard tunnel using STUN for NAT traversal, and if that direct path is blocked, traffic falls back to relay servers over TCP. The direct path is faster, so a healthy connection prefers it; relay is the reliable fallback when a firewall gets in the way.
The important consequence for firewalls is that every one of those connections is made outbound, from inside the network. Nothing needs to be opened inbound.
Required Outbound Ports
Section titled “Required Outbound Ports”The following ports must be allowed outbound from the network where the ARROW device is deployed:
Critical Services
Section titled “Critical Services”| Service | Destination | Protocol | Port(s) | Purpose |
|---|---|---|---|---|
| Management, signal, relay | [org].arrowvpn.io | TCP | 443 | Control plane, peer coordination, and encrypted relay fallback |
TCP 443 carries all three of these, so it is the one rule that is genuinely mandatory. A device with only this port open comes online and stays online, relaying its traffic.
P2P & NAT Traversal
Section titled “P2P & NAT Traversal”| Service | Destination | Protocol | Port(s) | Purpose |
|---|---|---|---|---|
| STUN / TURN | [org].arrowvpn.io | UDP | 3478 | NAT traversal for direct connections |
| WireGuard | [org].arrowvpn.io | UDP | 51820 | Direct encrypted data plane |
| WireGuard (peers) | Internet peers | UDP | 49152-65535 | Direct tunnel establishment between peers |
These are what turn a relayed connection into a direct one. They are a performance requirement, not a connectivity requirement.
Everything Else on the VPN Host
Section titled “Everything Else on the VPN Host”These exist but the device does not use them. Know them so you can answer a client who sees them in an egress log or a port scan, and so you do not ask for them by mistake:
| Protocol | Port | Service | Why the device does not use it |
|---|---|---|---|
| TCP | 3478 | STUN/TURN over TCP | coturn listens, but the host firewall does not accept it. The ARROW Manager VPN test still probes it, so a “blocked” result here is expected and not a fault. |
| TCP | 10000 | Signal, direct | Open on the host, but management.json points the client at TCP 443 for signal. |
| TCP | 33073 | Relay, direct | Open on the host, but management.json points the client at rels://[org].arrowvpn.io:443/relay. |
| UDP | 443 | HTTP/3 (QUIC) | Traefik runs with http3 on the websecure entrypoint, so a browser opening the dashboard may attempt QUIC. The host firewall does not accept UDP 443, so it silently falls back to TCP. This is the most likely origin of the old “UDP 443” line in this guide. |
| TCP | 80 | HTTP | Redirects to HTTPS and serves ACME certificate renewal. Server side only. |
| TCP | 22 | SSH | VTEM Labs administration. |
| UDP | 41641 | Tailscale | VTEM Labs administration path once public SSH is closed. |
Complete Port Summary
Section titled “Complete Port Summary”For IT teams, here’s the complete list to allow (replace [org] with your organization’s subdomain):
# Required[org].arrowvpn.io:443/tcp
# Preferred (direct tunnel instead of relay)[org].arrowvpn.io:3478/udp[org].arrowvpn.io:51820/udp0.0.0.0/0:49152-65535/udpSending this to a client’s IT team? Point them at Receiving a Device instead of pasting tables into an email. It is the same information written for someone who has never heard of ARROW, and it needs no account to read.
Common Blocking Scenarios
Section titled “Common Blocking Scenarios”A handful of network setups account for nearly all blocked connections. Match your symptoms to one of these to save yourself a lot of guessing.
1. Corporate Firewall Blocking Unknown Traffic
Section titled “1. Corporate Firewall Blocking Unknown Traffic”Symptoms:
- Device appears offline in ARROW Portal
- VPN shows “Disconnected” or “Connecting…” indefinitely
- Device works on home/mobile networks but not corporate
Cause: Egress firewall blocks outbound connections to unknown destinations
Solution: Request the IT team whitelist the NetBird endpoints listed above
2. Deep Packet Inspection (DPI) Blocking WireGuard
Section titled “2. Deep Packet Inspection (DPI) Blocking WireGuard”Symptoms:
- Initial connection appears to work, then drops
- Intermittent connectivity
- “Connection timeout” errors
Cause: DPI/IPS detects and blocks WireGuard protocol traffic
Solution:
- Request WireGuard protocol be allowed
- NetBird will fall back to relay over TCP/443 (looks like HTTPS)
- Ensure TCP relay ports are open
3. Web Proxy Intercepting Connections
Section titled “3. Web Proxy Intercepting Connections”Symptoms:
- SSL/TLS errors in logs
- “Certificate verification failed”
- Works when bypassing proxy
Cause: Transparent proxy intercepts and re-signs HTTPS traffic
Solution:
- Add the ARROW VPN endpoint to the proxy bypass list
- Configure a TLS inspection exception for
[org].arrowvpn.io
4. NAT/Firewall Blocking UDP
Section titled “4. NAT/Firewall Blocking UDP”Symptoms:
- Connection works but is slow
- High latency (>200ms) even on fast networks
- “Relay” indicator in NetBird status
Cause: UDP traffic is blocked, forcing TCP relay
Solution:
- Allow UDP 3478 and UDP 51820 outbound to
[org].arrowvpn.io - Direct P2P connections (UDP) are faster than relay (TCP)
5. Captive Portal / Network Authentication
Section titled “5. Captive Portal / Network Authentication”Symptoms:
- Device worked initially, stopped after network change
- “No internet” despite network connection
Cause: Network requires web authentication before allowing traffic
Solution:
- Complete captive portal authentication
- Use ethernet instead of guest WiFi when possible
Connected, But You Cannot Reach Any Devices
Section titled “Connected, But You Cannot Reach Any Devices”This one looks like a firewall problem and is not, so it is worth ruling out before you work through the port tables above.
Symptoms:
netbird statusreportsConnected, with a management URL and an assigned100.x.x.xaddress- The Console shows the devices you want as Online on the Peers tab
- Ping, SSH, and VNC to those devices all fail anyway
Cause: the workstation was enrolled with a setup key instead of being signed in through your identity provider.
Access on the ARROW VPN follows your identity, not your machine. When you connect by signing in through your IDP, the machine inherits the access attached to your account, which includes every device assigned to you. A setup key carries no identity. It enrolls the machine as standing infrastructure and places it in the key’s group, and that group is restrictive on purpose, so other peers can reach it and it cannot reach them. The result is a connection that comes up perfectly and can reach nothing.
Confirm it: open VPN in the Console and find your machine on the Peers tab. A workstation that authenticated through your IDP is listed against your user account and carries your groups. A machine that joined with a setup key shows the setup key’s group and no user.
Fix it: the group is decided when the machine enrolls, so changing it after the fact does not stick. Re-enroll instead:
-
In the Console, open VPN, find the machine on the Peers tab, and delete that peer.
-
On the workstation, disconnect the client:
Terminal window netbird down -
Bring it back up through your identity provider, with no
--setup-keyflag:Terminal window netbird up --management-url https://{your-domain}:443/Your browser opens; sign in exactly as you sign in to the Console.
-
Back in the Console, confirm on the Peers tab that the machine is now listed under your user account.
If the machine still cannot reach a device after re-enrolling under your account, the device is probably not assigned to you. Device access follows consultant assignments, so ask your organization administrator to assign it (see Network Access Control). If it is assigned and still unreachable, contact support.
Diagnostic Steps
Section titled “Diagnostic Steps”When the scenarios above do not pinpoint the problem, work through these steps to find out exactly what the network is letting through. The built-in test is the fast path; the manual commands are there when you want to see the raw results.
Use ARROW Manager VPN Connection Test (Recommended)
Section titled “Use ARROW Manager VPN Connection Test (Recommended)”ARROW Manager includes a built-in VPN Connection Test tool that checks connectivity to all required NetBird endpoints:
- Access ARROW Manager on the device
- Navigate to the Network or Diagnostics section
- Run the VPN Connection Test
- Review the results to see which endpoints are reachable and which are blocked
It tells you exactly which ports and endpoints the firewall is blocking, with no command line required. Start here before reaching for the manual steps.
Manual Diagnostic Steps
Section titled “Manual Diagnostic Steps”If you need to manually diagnose (via ARROW Manager terminal or SSH):
Step 1: Verify Basic Connectivity
Section titled “Step 1: Verify Basic Connectivity”From the ARROW device (via ARROW Manager terminal or SSH):
# Test internet connectivityping -c 4 8.8.8.8
# Test DNS resolution (use your organization's VPN address)nslookup [org].arrowvpn.io
# Test management APIcurl -I https://[org].arrowvpn.ioExpected results:
- Ping should succeed
- DNS should resolve to an IP
- Curl should return HTTP 200 or 401
Step 2: Test Required Ports
Section titled “Step 2: Test Required Ports”# Test TCP 443 to management, signal, and relaync -zv [org].arrowvpn.io 443
# Test UDP to STUN (requires netcat with UDP support)nc -zuv [org].arrowvpn.io 3478Expected: “Connection succeeded” or “open” for each
Step 3: Check NetBird Service Status
Section titled “Step 3: Check NetBird Service Status”# Check service statussudo systemctl status netbird
# View recent logssudo journalctl -u netbird -n 50 --no-pager
# Check connection statussudo netbird statusLook for:
- Service should be “active (running)”
- Status should show “Connected” with peer information
- Logs should not show repeated “connection failed” messages
Step 4: Test Direct vs Relay
Section titled “Step 4: Test Direct vs Relay”# Show detailed peer statussudo netbird status -dCheck the connection type:
direct= P2P connection established (optimal)relay= Traffic routing through relay (working but slower)disconnected= No connection (firewall blocking)
Step 5: Network Trace
Section titled “Step 5: Network Trace”If connections fail, capture what’s being blocked:
# Watch connection attempts (run as root)sudo tcpdump -i any host [org].arrowvpn.ioQuick Firewall Rule Request Template
Section titled “Quick Firewall Rule Request Template”Send this to the client’s IT team:
Subject: Firewall Rules Required for ARROW Device
We need the following outbound connections allowed for our ARROW device to establish VPN connectivity:
VPN Endpoint (replace [org] with your organization subdomain):
- [org].arrowvpn.io
Required:
- TCP 443 (management, signal, and encrypted relay)
Recommended, for a direct rather than relayed tunnel:
- UDP 3478 (NAT traversal)
- UDP 51820 (WireGuard data plane)
- UDP 49152-65535 to internet peers (direct tunnel establishment)
Notes:
- All connections are outbound only (no inbound required)
- Traffic is encrypted (WireGuard/TLS)
- Your specific VPN address was provided during onboarding
Fallback: Cellular Connection
Section titled “Fallback: Cellular Connection”If the client cannot or will not open required ports, ARROW devices include cellular failover:
- The device will automatically fall back to cellular if ethernet VPN fails
- Cellular connections bypass corporate firewalls entirely
- This is intended as a fallback, not primary connectivity
Device Not Online at All?
Section titled “Device Not Online at All?”If the device shows completely offline (not even on cellular), check the following:
1. Verify Device Power
Section titled “1. Verify Device Power”- Power Button: A small round button at the corner of the ARROW device, next to the HDMI port. One short press starts it.
- Indicator Lights: Two LEDs, red and blue, beside the PWR marking. Both come on as soon as the device has power:
- Lights off: No power - check the adapter, cable, and outlet
- Lights on, fan running: Powered and running
- Lights on, fan silent: Powered but never started - press the power button once
- The lights show power only. A device with the lights on and the fan off has power but has not started.
- This is rare, but devices may be shipped powered off or accidentally turned off
See Receiving a Device for a diagram of the button and the lights.
2. Check Cellular Signal
Section titled “2. Check Cellular Signal”If the device has power and is running but still not connecting via cellular:
- Weak Signal Area: The device may be in a location with poor cellular coverage
- Indoor Placement: Cellular signal can be weak in basements, server rooms, or buildings with thick walls
- Antenna Position: Ensure the device antenna (if external) is properly connected
Solutions:
- Move the device closer to a window or exterior wall
- Try a different location in the building
- Consider an external antenna extension if available
3. Fall Back to WiFi Hotspot
Section titled “3. Fall Back to WiFi Hotspot”If cellular signal is unavailable, you can configure the device to use WiFi:
- Access ARROW Manager via the device’s WiFi hotspot (during initial setup)
- Configure WiFi settings to connect to an available network
- The device will use WiFi for VPN connectivity
Connection Status Indicators
Section titled “Connection Status Indicators”In ARROW Portal
Section titled “In ARROW Portal”| Status | Meaning |
|---|---|
| Online | VPN connected, device healthy |
| Degraded | Connected via relay (slow) or intermittent |
| Offline | No VPN connection |
| Unknown | Waiting for first connection |
In ARROW Manager
Section titled “In ARROW Manager”| VPN Status | Meaning |
|---|---|
| Connected | Full connectivity established |
| Connecting | Attempting to establish connection |
| Disconnected | Not connected (check firewall) |
| Error | Service error (check logs) |
Escalation Path
Section titled “Escalation Path”If you’ve verified firewall rules are correct and connectivity still fails:
-
Collect diagnostics:
Terminal window sudo netbird status -d > /tmp/netbird-status.txtsudo journalctl -u netbird --since "1 hour ago" > /tmp/netbird-logs.txt -
Note the following:
- Device hostname (PVE-XXXX)
- Client network configuration (proxy, DPI, etc.)
- Time of last successful connection
- Error messages from logs
-
Contact VTEM Labs support with the above information
Related Documentation
Section titled “Related Documentation”- Receiving a Device - Client-facing network requirements to send to a site’s IT team
- VPN Management - VPN setup and configuration
- Device Management - Managing deployed devices
- ARROW Manager Troubleshooting - Device-specific issues