Skip to content

Commit 53005ab

Browse files
abidlabsXciDWauplinaliabid94
authored
Switch from SSH tunneling to FRP (#2509)
* FRP Poc (#2396) * FRP Poc * Gracefully handle exceptions in thread tunneling * comments * Fix share error message when files are built locally (#2502) * fix share error message * changelog * formatting * tunneling rename * version * formatting * remove test * changelog * version Co-authored-by: Abubakar Abid <[email protected]> Co-authored-by: Wauplin <[email protected]> * 2509 * updated url to testing.gradiodash.com * gradiotesting * format, version * gradio.live * temp fix for https * remove unnecessary tests * version * updated tunnel logic * formatting and tests * load testing * changes * Make private method + generate privilege key (#2519) * rm load test * frp * formatting * Update run.py * Update run.py * updated message * share=True * [DO NOT MERGE] Add pymux for FRP (#2747) * Add pymux for FRP * Cleaning pyamux * Cleaning pyamux + make it work * Forgot the thread * Reformat * some logs to be removed afterwards * added share to hello world * Transform into object * I guess it's cleaner now * Handle 404 + Transform to object * Fix params names * Add debug * windows fix Co-authored-by: Wauplin <[email protected]> Co-authored-by: Abubakar Abid <[email protected]> * removed share=True * formatting * hello world notebook * version * fixes * formatting * testing tunneling exists * tests * formatting * lint * Remove asyncio + kill proc on exit * version * version * update changelog * explicit message about reporting Co-authored-by: Adrien <[email protected]> Co-authored-by: Wauplin <[email protected]> Co-authored-by: Ali Abid <[email protected]>
1 parent 5182460 commit 53005ab

File tree

11 files changed

+152
-143
lines changed

11 files changed

+152
-143
lines changed

.gitignore

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,4 +44,10 @@ workspace.code-workspace
4444
*.h5
4545

4646
# log files
47-
.pnpm-debug.log
47+
.pnpm-debug.log
48+
49+
# Local virtualenv for devs
50+
.venv*
51+
52+
# FRP
53+
gradio/frpc_*

CHANGELOG.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,21 @@
11
# Upcoming Release
22

33
## New Features:
4-
No changes to highlight.
4+
5+
### New Shareable Links
6+
7+
Replaces tunneling logic based on ssh port-forwarding to that based on `frp` by [XciD](https://github.com/XciD) and [Wauplin](https://github.com/Wauplin) in [PR 2509](https://github.com/gradio-app/gradio/pull/2509)
8+
9+
You don't need to do anything differently, but when you set `share=True` in `launch()`,
10+
you'll get this message and a public link that look a little bit different:
11+
12+
```
13+
Setting up a public link... we have recently upgraded the way public links are generated. If you encounter any problems, please downgrade to gradio version 3.13.0
14+
.
15+
Running on public URL: https://bec81a83-5b5c-471e.gradio.live
16+
```
17+
18+
These links are a more secure and scalable way to create shareable demos!
519

620
## Bug Fixes:
721
* Allows `gr.Dataframe()` to take a `pandas.DataFrame` that includes numpy array and other types as its initial value, by [@abidlabs](https://github.com/abidlabs) in [PR 2804](https://github.com/gradio-app/gradio/pull/2804)
@@ -654,7 +668,6 @@ No changes to highlight.
654668
try to use `Series` or `Parallel` with `Blocks` by [@abidlabs](https://github.com/abidlabs) in [PR 2543](https://github.com/gradio-app/gradio/pull/2543)
655669
* Adds support for audio samples that are in `float64`, `float16`, or `uint16` formats by [@abidlabs](https://github.com/abidlabs) in [PR 2545](https://github.com/gradio-app/gradio/pull/2545)
656670

657-
658671
## Contributors Shoutout:
659672
No changes to highlight.
660673

demo/hello_world/run.ipynb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{"cells": [{"cell_type": "markdown", "id": 302934307671667531413257853548643485645, "metadata": {}, "source": ["# Gradio Demo: hello_world\n", "### The simplest possible Gradio demo. It wraps a 'Hello {name}!' function in an Interface that accepts and returns text.\n", " "]}, {"cell_type": "code", "execution_count": null, "id": 272996653310673477252411125948039410165, "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": 288918539441861185822528903084949547379, "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "\n", "def greet(name):\n", " return \"Hello \" + name + \"!\"\n", "\n", "demo = gr.Interface(fn=greet, inputs=\"text\", outputs=\"text\")\n", " \n", "demo.launch() "]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}
1+
{"cells": [{"cell_type": "markdown", "id": 302934307671667531413257853548643485645, "metadata": {}, "source": ["# Gradio Demo: hello_world\n", "### The simplest possible Gradio demo. It wraps a 'Hello {name}!' function in an Interface that accepts and returns text.\n", " "]}, {"cell_type": "code", "execution_count": null, "id": 272996653310673477252411125948039410165, "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": 288918539441861185822528903084949547379, "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "\n", "def greet(name):\n", " return \"Hello \" + name + \"!\"\n", "\n", "demo = gr.Interface(fn=greet, inputs=\"text\", outputs=\"text\")\n", " \n", "if __name__ == \"__main__\":\n", " demo.launch() "]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}

demo/hello_world/run.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ def greet(name):
55

66
demo = gr.Interface(fn=greet, inputs="text", outputs="text")
77

8-
demo.launch()
8+
if __name__ == "__main__":
9+
demo.launch()

gradio/blocks.py

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@
4444
from gradio.deprecation import check_deprecated_parameters
4545
from gradio.documentation import document, set_documentation_group
4646
from gradio.exceptions import DuplicateBlockError, InvalidApiName
47+
from gradio.tunneling import CURRENT_TUNNELS
4748
from gradio.utils import (
4849
TupleNoPrint,
4950
check_function_inputs_match,
@@ -1415,8 +1416,14 @@ def reverse(text):
14151416
raise RuntimeError("Share is not supported when you are in Spaces")
14161417
try:
14171418
if self.share_url is None:
1418-
share_url = networking.setup_tunnel(self.server_port, None)
1419-
self.share_url = share_url
1419+
print(
1420+
"\nSetting up a public link... we have recently upgraded the "
1421+
"way public links are generated. If you encounter any "
1422+
"problems, please report the issue and downgrade to gradio version 3.13.0\n."
1423+
)
1424+
self.share_url = networking.setup_tunnel(
1425+
self.server_name, self.server_port
1426+
)
14201427
print(strings.en["SHARE_LINK_DISPLAY"].format(self.share_url))
14211428
if not (quiet):
14221429
print(strings.en["SHARE_LINK_MESSAGE"])
@@ -1606,6 +1613,8 @@ def block_thread(
16061613
except (KeyboardInterrupt, OSError):
16071614
print("Keyboard interruption in main thread... closing server.")
16081615
self.server.close()
1616+
for tunnel in CURRENT_TUNNELS:
1617+
tunnel.kill()
16091618

16101619
def attach_load_events(self):
16111620
"""Add a load event for every component whose initial value should be randomized."""

gradio/networking.py

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
import uvicorn
1717

1818
from gradio.routes import App
19-
from gradio.tunneling import create_tunnel
19+
from gradio.tunneling import Tunnel
2020

2121
if TYPE_CHECKING: # Only import for type checking (to avoid circular imports).
2222
from gradio.blocks import Blocks
@@ -26,7 +26,7 @@
2626
INITIAL_PORT_VALUE = int(os.getenv("GRADIO_SERVER_PORT", "7860"))
2727
TRY_NUM_PORTS = int(os.getenv("GRADIO_NUM_PORTS", "100"))
2828
LOCALHOST_NAME = os.getenv("GRADIO_SERVER_NAME", "127.0.0.1")
29-
GRADIO_API_SERVER = "https://api.gradio.app/v1/tunnel-request"
29+
GRADIO_API_SERVER = "https://api.gradio.app/v2/tunnel-request"
3030

3131

3232
class Server(uvicorn.Server):
@@ -157,14 +157,15 @@ def start_server(
157157
return server_name, port, path_to_local_server, app, server
158158

159159

160-
def setup_tunnel(local_server_port: int, endpoint: str) -> str:
161-
response = requests.get(
162-
endpoint + "/v1/tunnel-request" if endpoint is not None else GRADIO_API_SERVER
163-
)
160+
def setup_tunnel(local_host: str, local_port: int) -> str:
161+
response = requests.get(GRADIO_API_SERVER)
164162
if response and response.status_code == 200:
165163
try:
166164
payload = response.json()[0]
167-
return create_tunnel(payload, LOCALHOST_NAME, local_server_port)
165+
remote_host, remote_port = payload["host"], int(payload["port"])
166+
tunnel = Tunnel(remote_host, remote_port, local_host, local_port)
167+
address = tunnel.start_tunnel()
168+
return address
168169
except Exception as e:
169170
raise RuntimeError(str(e))
170171
else:
@@ -174,11 +175,11 @@ def setup_tunnel(local_server_port: int, endpoint: str) -> str:
174175
def url_ok(url: str) -> bool:
175176
try:
176177
for _ in range(5):
177-
time.sleep(0.500)
178178
with warnings.catch_warnings():
179179
warnings.filterwarnings("ignore")
180180
r = requests.head(url, timeout=3, verify=False)
181181
if r.status_code in (200, 401, 302): # 401 or 302 if auth is set
182182
return True
183+
time.sleep(0.500)
183184
except (ConnectionError, requests.exceptions.ConnectionError):
184185
return False

gradio/tunneling.py

Lines changed: 98 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -1,101 +1,98 @@
1-
"""
2-
This file provides remote port forwarding functionality using paramiko package,
3-
Inspired by: https://github.com/paramiko/paramiko/blob/master/demos/rforward.py
4-
"""
5-
6-
import select
7-
import socket
8-
import sys
9-
import threading
10-
import warnings
11-
from io import StringIO
12-
13-
from cryptography.utils import CryptographyDeprecationWarning
14-
15-
with warnings.catch_warnings():
16-
warnings.filterwarnings("ignore", category=CryptographyDeprecationWarning)
17-
import paramiko
18-
19-
20-
def handler(chan, host, port):
21-
sock = socket.socket()
22-
try:
23-
sock.connect((host, port))
24-
except Exception as e:
25-
verbose(f"Forwarding request to {host}:{port} failed: {e}")
26-
return
27-
28-
verbose(
29-
"Connected! Tunnel open "
30-
f"{chan.origin_addr} -> {chan.getpeername()} -> {(host, port)}"
31-
)
32-
33-
while True:
34-
r, w, x = select.select([sock, chan], [], [])
35-
if sock in r:
36-
data = sock.recv(1024)
37-
if len(data) == 0:
38-
break
39-
chan.send(data)
40-
if chan in r:
41-
data = chan.recv(1024)
42-
if len(data) == 0:
43-
break
44-
sock.send(data)
45-
chan.close()
46-
sock.close()
47-
verbose(f"Tunnel closed from {chan.origin_addr}")
48-
49-
50-
def reverse_forward_tunnel(server_port, remote_host, remote_port, transport):
51-
transport.request_port_forward("", server_port)
52-
while True:
53-
chan = transport.accept(1000)
54-
if chan is None:
55-
continue
56-
thr = threading.Thread(target=handler, args=(chan, remote_host, remote_port))
57-
thr.setDaemon(True)
58-
thr.start()
59-
60-
61-
def verbose(s, debug_mode=False):
62-
if debug_mode:
63-
print(s)
64-
65-
66-
def create_tunnel(payload, local_server, local_server_port):
67-
client = paramiko.SSHClient()
68-
client.set_missing_host_key_policy(paramiko.WarningPolicy())
69-
70-
verbose(f'Conecting to ssh host {payload["host"]}:{payload["port"]} ...')
71-
try:
72-
with warnings.catch_warnings():
73-
warnings.simplefilter("ignore")
74-
client.connect(
75-
hostname=payload["host"],
76-
port=int(payload["port"]),
77-
username=payload["user"],
78-
pkey=paramiko.RSAKey.from_private_key(StringIO(payload["key"])),
79-
)
80-
except Exception as e:
81-
print(f'*** Failed to connect to {payload["host"]}:{payload["port"]}: {e}')
82-
sys.exit(1)
83-
84-
verbose(
85-
f'Now forwarding remote port {payload["remote_port"]}'
86-
f"to {local_server}:{local_server_port} ..."
87-
)
88-
89-
thread = threading.Thread(
90-
target=reverse_forward_tunnel,
91-
args=(
92-
int(payload["remote_port"]),
93-
local_server,
94-
local_server_port,
95-
client.get_transport(),
96-
),
97-
daemon=True,
98-
)
99-
thread.start()
100-
101-
return payload["share_url"]
1+
import atexit
2+
import os
3+
import platform
4+
import re
5+
import subprocess
6+
from typing import List
7+
8+
VERSION = "0.1"
9+
CURRENT_TUNNELS: List["Tunnel"] = []
10+
11+
12+
class Tunnel:
13+
def __init__(self, remote_host, remote_port, local_host, local_port):
14+
self.proc = None
15+
self.url = None
16+
self.remote_host = remote_host
17+
self.remote_port = remote_port
18+
self.local_host = local_host
19+
self.local_port = local_port
20+
21+
@staticmethod
22+
def download_binary():
23+
machine = platform.machine()
24+
if machine == "x86_64":
25+
machine = "amd64"
26+
27+
# Check if the file exist
28+
binary_name = f"frpc_{platform.system().lower()}_{machine.lower()}"
29+
binary_path = os.path.join(os.path.dirname(__file__), binary_name)
30+
31+
extension = ".exe" if os.name == "nt" else ""
32+
33+
if not os.path.exists(binary_path):
34+
import stat
35+
36+
import requests
37+
38+
binary_url = f"https://cdn-media.huggingface.co/frpc-gradio-{VERSION}/{binary_name}{extension}"
39+
resp = requests.get(binary_url)
40+
41+
if resp.status_code == 403:
42+
raise OSError(
43+
f"Cannot set up a share link as this platform is incompatible. Please "
44+
f"create a GitHub issue with information about your platform: {platform.uname()}"
45+
)
46+
47+
resp.raise_for_status()
48+
49+
# Save file data to local copy
50+
with open(binary_path, "wb") as file:
51+
file.write(resp.content)
52+
st = os.stat(binary_path)
53+
os.chmod(binary_path, st.st_mode | stat.S_IEXEC)
54+
55+
return binary_path
56+
57+
def start_tunnel(self) -> str:
58+
binary_path = self.download_binary()
59+
self.url = self._start_tunnel(binary_path)
60+
return self.url
61+
62+
def kill(self):
63+
if self.proc is not None:
64+
print(f"Killing tunnel {self.local_host}:{self.local_port} <> {self.url}")
65+
self.proc.terminate()
66+
self.proc = None
67+
68+
def _start_tunnel(self, binary: str) -> str:
69+
CURRENT_TUNNELS.append(self)
70+
command = [
71+
binary,
72+
"http",
73+
"-n",
74+
"random",
75+
"-l",
76+
str(self.local_port),
77+
"-i",
78+
self.local_host,
79+
"--uc",
80+
"--sd",
81+
"random",
82+
"--ue",
83+
"--server_addr",
84+
f"{self.remote_host}:{self.remote_port}",
85+
"--disable_log_color",
86+
]
87+
88+
self.proc = subprocess.Popen(
89+
command, stdout=subprocess.PIPE, stderr=subprocess.PIPE
90+
)
91+
atexit.register(self.kill)
92+
url = ""
93+
while url == "":
94+
line = self.proc.stdout.readline()
95+
line = line.decode("utf-8")
96+
if "start proxy success" in line:
97+
url = re.search("start proxy success: (.+)\n", line).group(1)
98+
return url

gradio/version.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
3.13.0
1+
3.13.0

requirements.txt

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@ matplotlib
77
numpy
88
orjson
99
pandas
10-
paramiko
1110
pillow
1211
pycryptodome
1312
python-multipart

test/test_networking.py

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
"""Contains tests for networking.py and app.py"""
22

33
import os
4-
import unittest.mock as mock
54
import urllib
65
import warnings
76

@@ -75,11 +74,6 @@ def test_start_server(self):
7574

7675

7776
class TestURLs:
78-
def test_setup_tunnel(self):
79-
networking.create_tunnel = mock.MagicMock(return_value="test")
80-
res = networking.setup_tunnel(None, None)
81-
assert res == "test"
82-
8377
def test_url_ok(self):
8478
res = networking.url_ok("https://www.gradio.app")
8579
assert res

0 commit comments

Comments
 (0)