How do I log in to the token method?

Asked 2 years ago, Updated 2 years ago, 35 views

For personal interest,

To the website using tokens I wanted to log in and try scraping.
I wrote it down, but it doesn't work after logging in.
I tried my best to find out what caused it, but I'm not sure.

Here is the code:

Data[_Token][key] and data[_Token][fields] are retrieved by manually searching the page.
I know everything else, so I input four values.
I checked the chrome function, but I'm not sure if it's correct.

The results return HTML content indicating that you have been redirected to the login failure page.

I only have a few months of JAVA experience. The web-related part lacks a lot of knowledge, so
There may have been a fatal error in the code...
I would appreciate it if you could let me know.

public static void main(String[]args)throws Exception {


    String USER_AGENT = "Mozilla/5.0";
    String url="https://www.lancers.jp/user/login";
    String user="https://www.lancers.jp/mypage";
    HttpGet=new HttpGet(url);
    HttpPost post = new HttpPost(url);

    RequestConfig requestconf = RequestConfig.custom()
            .setCircularRedirectsAllowed(true)
            .setRedirectsEnabled (true)
            .build();
    List<Header>headers=new ArrayList<Header>();
    headers.add(new BasicHeader("User-Agent", USER_AGENT));
    heads.add(new BasicHeader("Accept-Charset", "utf-8"));
    headers.add(new BasicHeader("Accept-Language", "ja,en;q=0.8"));
    headers.add(new BasicHeader("Content-Type", "application/x-www-form-urlencoded"));

    CloseableHttpClientclient=HttpClientBuilder.create()
            .setRedirectStratey (new LaxRedirectStratey())
            .setDefaultRequestConfig(reqconf)
            .setDefaultHeaders (headers)
            .build();

    CloseableHttpResponse loginpage=client.execute(get);
    String loginpageHTML = EntityUtils.toString(loginpage.getEntity(), "UTF-8");
    // Even if it's Jsoup or something like that, it seems impossible to parse Token only, so I searched manually...
    Pattern p=Pattern.compile("(?is)value=\"(.+?)\"); 
    Matcher m=p.matcher (loginpageHTML);
    // Look for tokens.
    String [ ] TokenArray=new String[6];

    inti = 0;
    while(m.find()){
        TokenArray[i]=m.group((1));
        i++;
    }
    // TokenArray[1] TokenArray[5]
    // 1 is key, 5 is fields. Let's include this in the POST transmission.

    List<NameValuePair>params=new ArrayList<NameValuePair>();;
    params.add(new BasicNameValuePair("data[_Token][key])", TokenArray[1]);
    params.add(new BasicNameValuePair("data[User][email]", "My ID"));
    params.add(new BasicNameValuePair("data[User][password]", "My PASS"));
    param.add(new BasicNameValuePair("data[User][auto_login]", "0"));
    param.add(new BasicNameValuePair("data[User][refer]", "http://www.lancers.jp/"));
    params.add(new BasicNameValuePair("data[_Token][fields]", TokenArray[5])));

    post.setHeader("Connection", "keep-alive");
    post.setHeader("Refer", user);
    post.setHeader("Content-Type", "application/x-www-form-urlencoded";

    post.setEntity (new UrlEncodedFormEntity (params));

    String html="";
    CloseableHttpResponse response=client.execute(post);

    try{
    HttpEntity entity=response.getEntity();
    html=EntityUtils.toString(entity, "UTF-8");
    EntityUtils.consume(entity);
    }finally {
        response.close();
    }

    Document doc = Jgroup.parse(html);
    System.out.println(doc);


}

}

java

2022-09-30 19:35

1 Answers

I checked the source code. It seems to be working for now.
When you do this, you can log in with your browser and look at the request header from the browser developer mode.However, I don't think the server is providing the API, so it's kind of like reverse engineering.I think it would be better to practice with a web service where the server publishes such an API.Also, this is the first time I've heard the word token method, but is it a unique term?

Now, from the log, you can see that the login flow should be as follows:

 1.POST https://www.lancers.jp/user/login

   *** Send Form to Receive Cookie CakeCookie [User]***

   Send Form
   data[_Token] [key]—Token
   data [User] [email]—Email address
   data [User] [password]—Password
   data [User] [ auto_login ]—Unknown value
   data [User] [refer]:reference
   submit —Log in (send button value:probably optional)
   data[_Token][fields]:URL-encoded tokens

   [Send cookies]
   CakeCookie [LANCERS_C]
   CakeCookie [CLIENT]
   CakeCookie [arcmnd]

   [Cookies that can be received]
   CakeCookie [User]

2. GET http://www.lancers.jp/mypage

   *** Include CakeCookie [Users] in the request header to get the location of GET,

   [Receiveable Headers]
   Location: https://www.lancers.jp/mypage?[0-9]{10}&

3. GET https://www.lancers.jp/mypage?[0-9]{10}&

   *** Log in to mypage with HTTPS***
Remote Address:52.68.161.170:443
Request URL: https://www.lancers.jp/user/login
Request Method: POST
Status Code: 302 Found
Accept:text/html, application/xhtml+xml, application/xml;q=0.9, image/webp, */*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: ja, en-US; q = 0.8, en; q = 0.6
Cache-Control: max-age = 0
Connection:keep-alive
Content-Length: {*** Length of content to POST***}
Content-Type: application/x-www-form-urlencoded
Cookies: {***Cookies Various***}
Host: www.lancers.jp
Origin: https://www.lancers.jp
Referer: https://www.lancers.jp/user/login?{***User Unique ID (Number)***}&
User-Agent: {***Anything here***}
_method:POST
data[_Token][key]: token_token_token_token_token...
data [User] [email]: [email protected]
data [User] [password]—this_is_password
data[User][auto_login]—0 // 0 and 1 are configured on the log
data[User][auto_login]: 1 // seems to be meaningless
data [User] [refer]—http://www.lancers.jp/
submit —Log in
data[_Token][fields]: {***URL encoded token***}
Connection:keep-alive
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Mon, 22 Jun 2015 01:02:31 GMT
Location: http://www.lancers.jp/mypage
P3P: CP = "NOI ADM DEV PSAi COM NAV OUROTRo STP INDEM"
Server: Apache
Set-Cookie: CakeCookie [User]     
Set-Cookie: CakeCookie [LANCERS_C]
Set-Cookie: CakeCookie [CLIENT]   
Set-Cookie: CakeCookie [arcmnd]   

Get the actual location of mypage.For browsers, if location is included, it will automatically redirect.

Location: https://www.lancers.jp/mypage?[0-9]{10}&

Request via HTTPS and complete with this


2022-09-30 19:35

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.