Skip to content

Implement hCaptcha solving via 2Captcha [URGENT MERGE] #222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 27, 2025

Conversation

gohoski
Copy link
Contributor

@gohoski gohoski commented Jan 6, 2025

This PR finally implements hCaptcha solving via paid service 2Captcha and browser automation library Playwright with rebrowser-patches. If the CAPTCHA verification is needed by Suno, it automatically spawns a browser, goes to suno.com/create, solves the CAPTCHA and gets the hCaptcha token. No any 'fake app' songs have ever been generated in my tests.
Unfortunately, no free solutions to this problem exist. While this solution is resource-intensive, could take a really long time (from 100s to 500s, depends on the worker that solves the CAPTCHA) and not free, it actually works and you don't actually have to pay much. hCaptcha tokens also last anything from 20 mins up to 2 hours, so if you have a Suno account with a subscription that enhances the limits, you won't have to solve CAPTCHAs that much if you're lucky enough to get a 2-hours hCaptcha token.

Besides that, this PR:

  • implements sunoApi instance caching so sessions won't be constantly updated
  • adds support for entering cookies not only in SUNO_COOKIE, but also in the Cookie HTTP header, which can really be useful for using multiple accounts
  • update docs and add Russian docs. I also would like to ask Chinese speakers to update their docs, since now they will be outdated

However, before merging this PR, I highly recommend you to test it. Suno and hCaptcha can change something in their CAPTCHA really soon and this code could not work. But, with the current solution, the possibility of that happening is still low.

I would also like to ask @blueeon to remove the Chinese paid Suno API implementations spam in the Issues tab. It is really annoying and not helping the current situation.

Also, maybe you could create a Discord server or a Telegram group-chat for suno-api, so we would get notified about problems more fast and just talk about Suno things. It would be a very useful addition, especially when Suno is constantly taking down external APIs.

Fixes #220, fix #215, fix #206, fix #205, fix #204, fix #200, fix #198, fix #195 (now that's a LOT of issues)

- implement hCaptcha solving via paid service 2Captcha and browser automation library Playwright with rebrowser-patches

- implement sunoApi instances caching so sessions won't be constantly updated

- add support for entering cookies not only in SUNO_COOKIE, but also the Cookie HTTP header

- update docs and add Russian docs
Copy link

vercel bot commented Jan 6, 2025

@gohoski is attempting to deploy a commit to the Linkly AI LLC's projects Team on Vercel.

A member of the Team first needs to authorize it.

@gohoski
Copy link
Contributor Author

gohoski commented Jan 6, 2025

Also, this PR imitates requests from the Android app. This way we don't have to solve Cloudflare Turnstile and don't need to constantly update this.currentToken

also change the Suno interface load trigger from a locator to waitForResponse since the previous locator could not exist on a Suno account without any songs
please note that you can't increase the timeout in any way, even by clicking,so the only option we have is to just reinstate the solving process
@GrisBouille
Copy link

I tried this PR on my own 2 days ago (with the help of Gohoski) and it's working great.
2captcha paid service is only 3$ for 1000 captcha solving and never failed so far.
Many thanks to Gohoski for this fix !

@gohoski
Copy link
Contributor Author

gohoski commented Jan 8, 2025

If you have any problems with this PR, you can say them here or DM me via Discord/Telegram in my bio.

@angejr
Copy link

angejr commented Jan 10, 2025

I occasionally ( Every 2/3 generations) get a unhandled TimeoutError caused by " await page.waitForResponse('https://img**.hcaptcha.com/**', { timeout: 60000 }); // wait for hCaptcha image to load"
Do you have any idea what the issue is ?

@gohoski
Copy link
Contributor Author

gohoski commented Jan 10, 2025

@angejr can you try with BROWSER_HEADLESS=false and see how the hCaptcha window looks? Maybe it's a new hCaptcha type

@angejr
Copy link

angejr commented Jan 10, 2025

Somehow removing headless makes it work, thanks !

- fixed bug in dragging type of hCaptcha when worker did not select an even amount of coordinates and it would crash
- change waitForResponse function to a waitForRequests util function with more proper checks
@gohoski
Copy link
Contributor Author

gohoski commented Jan 10, 2025

@angejr try it now with headless mode, I tried to change the waitForRequests logic

@hedleyroos
Copy link

@gohoski I've done some work on getting the branch to work in Docker. I tried getting it to work with Alpine, as this is the base in the current Dockerfile, but without success. It works headless with Debian Bookworm base, and Microsoft's official Playwright base image that's based on Ubuntu. In all cases I cannot get non-headless mode to work under Docker. I'll summarize what I needed to do:

Change Dockerfile to use:

FROM mcr.microsoft.com/playwright:v1.49.1-noble

Change src/lib/SunoApi.ts to launch the browser with:

--enable-unsafe-swiftshader --disable-gpu

@jonico
Copy link
Contributor

jonico commented Jan 20, 2025

@gohoski: thank you for your contribution ❤️. After fixing some lock file issues with Vercel, I get the following error when executing an API call:

error": "Error: pOST: Executable doesn't exist at /home/sbx_user1051/.cache/ms-playwright/chromium_headless_shell-1148/chrome-linux/headless_shell\n╔═════════════════════════════════════════════════════════════════════════╗\n║ Looks like Playwright Test or Playwright was just installed or updated. ║\n║ Please run the following command to download new browsers:              ║\n║                                                                         ║\n║     npx playwright install                                              ║\n║                                                                         ║\n║ <3 Playwright Team                                                      ║\n╚═════════════════════════════════════════════════════════════════════════╝"

Is there a way to tell Vercel it needs playwright installed?

@hedleyroos
Copy link

@jonico I don't use Vercel, but I guess it uses Docker. I had to use a different Dockerfile because I had the same issue:

# syntax=docker/dockerfile:1 
                                                                                                                                                                                                       
FROM node:lts-bookworm AS builder                                                                                       
WORKDIR /src                                                                                                            
COPY package*.json ./                                                                                                   
RUN npm install                                                                                                         
COPY . .                                                                                                                
RUN npm run build                                                                                                       
                                                                                                                    
FROM node:lts-bookworm                                                                                                  
WORKDIR /app                                                                                                            
COPY package*.json ./                                                                                                   
                                                                                                                    
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y libnss3 \                                       
    libdbus-1-3 libatk1.0-0 libatk-bridge2.0-0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 \                       
    libgbm1 libxkbcommon0 libasound2 libcups2 xvfb                                                                      
                                                                                                                    
ARG SUNO_COOKIE                                                                                                         
ARG BROWSER                                                                                                             
RUN if [ -z "$SUNO_COOKIE" ]; then echo "Warning: SUNO_COOKIE is not set"; fi                                           
ENV SUNO_COOKIE=${SUNO_COOKIE}                                                                                          
RUN if [ -z "$BROWSER" ]; then echo "Warning: BROWSER is not set; will use chromium by default"; fi                     
ENV BROWSER=${BROWSER:-chromium}                                                                                        
                                                                                                                    
RUN npm install --only=production                                                                                       
                                                                                                                    
# Install all supported browsers, else switching browsers requires an image rebuild                                     
RUN npx playwright install chromium                                                                                     
#RUN npx playwright install firefox                                                                                     
                                                                                                                    
COPY --from=builder /src/.next ./.next                                                                                  
EXPOSE 3000                                                                                                             
CMD ["npm", "run", "start"]

I also had to change src/lib/SunoApi.ts for Playwright browser launch options so we can run headless in a container:

 '--enable-unsafe-swiftshader',
 '--disable-gpu',
 '--disable-setuid-sandbox'

This was the only way I could get Docker to work. Maybe this is of use for Vercel.

FWIW I also changed docker-compose slightly:

env_file: ".env"

…ead of a manual command, fix Docker & add notice about macOS recommendation
@gohoski
Copy link
Contributor Author

gohoski commented Jan 21, 2025

Updated Docker (thanks to @hedleyroos!) in my new commit and added a cookie check, so unnecessary cookies from some browsers won't be considered as a full SUNO_COOKIE. Please test this and LMK.

Interestingly, I discovered that you can actually use a macOS system to vastly improve performance and to avoid CAPTCHAs. This is because it is an unpopular platform for web-scraping, especially when you put it next to Linux & Windows. Also put that in the README. Better not let Ace Data Cloud to know about this xD

Also, I made it so the browser would be installed automatically on npm install by using the @playwright/browser-chromium package. This should fix the error in Vercel.

@jonico
Copy link
Contributor

jonico commented Jan 22, 2025

This should fix the error in Vercel.

@gohoski thank you for your efforts 🎉 - unfortunately, same issue in Vercel (no chromium browser found), as the runtime environment (serverless) is different from the build environment (where chromium gets installed)

@gohoski
Copy link
Contributor Author

gohoski commented Jan 26, 2025

@jonico Try to npm install @sparticuz/[email protected] and apply this patch.

If it still doesn't work, we will have to unfortunately discontinue using Vercel. I have also heard that it has a 60s limit on each request, which is no bueno for suno-api. Maybe we can develop a job/task queue system for these cases, but first we will need to discuss is Vercel even a good host for these projects.
sigma.patch

@artokun
Copy link

artokun commented Jan 26, 2025

Don't give up, it's a great PR. I actually have a working chromium vercel deployment for another app I made for scraping substack posts on the edge using playwright , I will bring that over and share my solution once I have it working

@jonico
Copy link
Contributor

jonico commented Jan 27, 2025

If it still doesn't work, we will have to unfortunately discontinue using Vercel. I have also heard that it has a 60s limit on each request, which is no bueno for suno-api. Maybe we can develop a job/task queue system for these cases, but first we will need to discuss is Vercel even a good host for these projects.

@gohoski I am now getting a new error message:

TimeoutError: pOST: Timeout 30000ms exceeded.\nCall log:\n  - waiting for locator('.custom-textarea')\n"

Looks as if the approach may work in general.

Maybe we can develop a job/task queue system for these cases, but first we will need to discuss is Vercel even a good host for these projects.

You can change the max timeout of Vercel in the free tier to 60 seconds (see #86 ) and up to 900 seconds in the paid plans.

@blueeon
Copy link
Contributor

blueeon commented Jan 27, 2025

Thank you for your work. I will merge it as soon as possible.
I have already deleted those spam.

@blueeon blueeon merged commit 48d667b into gcui-art:main Jan 27, 2025
1 check failed
@artokun
Copy link

artokun commented Jan 30, 2025

Don't give up, it's a great PR. I actually have a working chromium vercel deployment for another app I made for scraping substack posts on the edge using playwright , I will bring that over and share my solution once I have it working

Fixed in #227

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment