开发者

When you scrape at any real volume, the bottleneck is rarely your code — it's the target site's rate limiting and IP bans. Rotating residential proxies solve this by routing each request through a different real-user IP. Here's how to wire them into requests and Scrapy in Python, including the sticky-session trick most tutorials skip. The proxy URL format A residential proxy is just an authenticated HTTP/SOCKS endpoint. With a pool gateway, you target a country and control session behavior through the username , not separate endpoints: http://USERNAME_country-us_session-a1b2c3_lifetime-30:PASSWORD@proxy.gproxy.net:1000 country-us — exit country (ISO code) session-a1b2c3 — a sticky-session id; reuse it to keep the same IP , change it to rotate lifetime-30 — how many minutes that session's IP stays fixed Basic request through a rotating proxy import requests USER = " USERNAME " PWD = " PASSWORD " def proxy ( country = " us " , session = None , lifetime = 30 ): tag = f " _country- { country } " if session : tag += f " _session- { session } _lifetime- { lifetime } " url = f " http:// { USER }{ tag } : { PWD } @proxy.gproxy.net:1000 " return { " http " : url , " https " : url } # New IP on every call (no session id): r = requests . get ( " https://api.ipify.org?format=json " , proxies = proxy (), timeout = 30 ) print ( r . json ()[ " ip " ]) Run it in a loop and you'll see a different IP each time — the gateway rotates automatically when no session id is present. Sticky sessions: keep one IP across requests Some flows (login, multi-step checkouts, paginated results behind a cookie) break if your IP changes mid-session. Pin the IP by passing a stable session id: import uuid sess = uuid . uuid4 (). hex [: 8 ] # one id for the whole flow p = proxy ( country = " de " , session = sess , lifetime = 30 ) s = requests . Session () s . proxies . update ( p ) s . get ( " https://example.com/login " ) s . post ( " https://example.com/login " , data = {...}) # same exit IP When you

I spent last month reviewing JWT implementations across 12 open-source Node.js projects on GitHub — ranging from starter templates with 2k stars to production boilerplates used by teams at real companies. I found the same 6 mistakes in almost every one. None of these projects are bad. The developers are skilled. The mistakes are subtle, copy-paste errors from tutorials that nobody questioned. Here they are. Mistake 1 — The Secret Is Literally "secret" I found this in three separate projects: const token = jwt . sign ({ userId : user . id }, " secret " , { expiresIn : " 1h " }); This secret is in every JWT tutorial on the internet. It is in the jwt.io documentation. It is in the jsonwebtoken README. Developers copy it and forget to replace it. A 6-character ASCII secret has approximately 42 bits of entropy. A GPU cluster cracks it from a dictionary in milliseconds. Generate a real secret here — it takes 3 seconds and produces a 256-bit cryptographically random key. Mistake 2 — jwt.decode() Used in Auth Middleware // DANGEROUS — this is in a production auth middleware const decoded = jwt . decode ( req . headers . authorization . split ( " " )[ 1 ]); if ( ! decoded . userId ) return res . status ( 401 ). send ( " Unauthorized " ); jwt.decode() does not verify the signature. It reads the payload regardless of whether the token is valid, expired, or forged. An attacker can craft any payload they want and it will pass this check. The fix is two characters: jwt.verify() . const decoded = jwt . verify ( token , process . env . JWT_SECRET , { algorithms : [ " HS256 " ] }); Mistake 3 — Algorithm Not Specified in verify() // Missing algorithms option jwt . verify ( token , secret ); Without { algorithms: ['HS256'] } , the library trusts whatever algorithm is in the token's header. An attacker can create a token with alg: none and an empty signature — and jwt.verify() will accept it. Always specify the expected algorithm explicitly. Mistake 4 — Secret Committed to Version Cont

submitted by /u/vitiral [link] [留言]

submitted by /u/xoner2 [link] [留言]