Curl SpringWebClient:如何跳过/接受Cookie同意请求?

Curl SpringWebClient:如何跳过/接受Cookie同意请求?,curl,postman,webclient,spring-webflux,cookieconsent,Curl,Postman,Webclient,Spring Webflux,Cookieconsent,我正在使用SpringWebClient检索。 它工作得很好,但现在他们在尝试访问JSON数据时添加了cookie同意模式 从技术上讲,这是我从Chrome上看到的网络流: 得到 卷曲'https://www.forbes.com/forbesapi/org/global2000/2020/position/true.json?limit=2000“ -H‘权威:www.forbes.com’ -H'pragma:无缓存' -H'缓存控制:无缓存' -H'dnt:1' -H'升级不安全请求:1'

我正在使用SpringWebClient检索。 它工作得很好,但现在他们在尝试访问JSON数据时添加了cookie同意模式

从技术上讲,这是我从Chrome上看到的网络流:

  • 得到

    卷曲'https://www.forbes.com/forbesapi/org/global2000/2020/position/true.json?limit=2000“
    -H‘权威:www.forbes.com’
    -H'pragma:无缓存'
    -H'缓存控制:无缓存'
    -H'dnt:1'
    -H'升级不安全请求:1'
    -H'用户代理:Mozilla/5.0(X11;Linux x86_64)AppleWebKit/537.36(KHTML,如Gecko)Chrome/83.0.4103.61 Safari/537.36'
    -接受:text/html,application/xhtml+xml,application/xml;q=0.9,图像/webp,图像/apng,/;q=0.8,申请/签名交换;v=b3;q=0.9'
    -H'秒获取站点:无'
    -H'秒提取模式:导航'
    -H'秒获取用户:?1'
    -H'秒获取目的地:文档'
    -接受语言:它,它;q=0.9,en GB;q=0.8,en;q=0.7,ru-ru;q=0.6,ru;q=0.5,en-US;q=0.4'
    --压缩的

  • 302重定向到

  • 这是同意模式
  • 寄往 单击“接受”(Accetto)时
  • 在网络中,我看不到更多重定向,但提供了初始JSON:
  • 使用Spring WebClient是否有办法跳过/绕过/接受这样的同意? 一个卷发或邮递员的例子也就足够了

    更新

    这是返回JSON的GET。它有一些饼干,所以也许这样它可以工作。但是,我需要将其自动化

    curl 'https://www.forbes.com/forbesapi/org/global2000/2020/position/true.json?limit=2000' \
      -H 'authority: www.forbes.com' \
      -H 'pragma: no-cache' \
      -H 'cache-control: no-cache' \
      -H 'dnt: 1' \
      -H 'upgrade-insecure-requests: 1' \
      -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36' \
      -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
      -H 'sec-fetch-site: none' \
      -H 'sec-fetch-mode: navigate' \
      -H 'sec-fetch-user: ?1' \
      -H 'sec-fetch-dest: document' \
      -H 'accept-language: it-IT,it;q=0.9,en-GB;q=0.8,en;q=0.7,ru-RU;q=0.6,ru;q=0.5,en-US;q=0.4' \
      -H 'cookie: notice_behavior=expressed,eu; notice_gdpr_prefs=0,1,2:1a8b5228dd7ff0717196863a5d28ce6c; notice_preferences=2:1a8b5228dd7ff0717196863a5d28ce6c; client_id=8ec8eb2817c5d8913fe8f17384e17644a7e; global_ad_params=%7B%7D; __aaxsc=0; _ga=GA1.2.1634735456.1605525570; _gid=GA1.2.1103122519.1605525570; _fbp=fb.1.1605525570288.36176089; crdl_forbes-livebID=aa58b21d-aa96-4ee0-8340-fb48af0c1a94; crdl_forbes-liveaID=anonimIcgdCVIkNG7BUNba5etK; rbzid=l/U6Jw1r1yWT74cdJuQgosf28tekVPrGz9xIBrFnVQIm5sDTTvTWn/1+r8KTgAdbwgwm63USRoqfI+JwWc3BgQoGTMvyrKX2VRjibHRVHjUSGWqZKX+XhkPhAl2YVYGXbbgLmximPCVub7cvx/UClbnBmD33n1qHOtWLY9urqrdhtD5OGF4BLBZ0L5m0YGx1EKQb+AkPBgt9hT3pryTWI4S84CzcPLIsJTmCtn7YRLgplAvwGe1EA/W8a90o6dER+bVJ+Sy/8/dWswABzRNVwQ==; rbzsessionid=d40c35ba621223231fd16c42b943f00a; __pnahc=0; __tbc=%7Bjzx%7DIFcj-ZhxuNCMjI4-mDfH1NUCZW7-CmMn43P4peooi3pN7s1U6SLs77TJd0_MaIDk7egQVHeoSjUK2k3sjldowD8Ff5A-5DxOArBhtCY3kQmc3ozP8vUOOShHwL0JdQDA3E9l5ifC0NcYbET3aSZxuA; __pat=-18000000; _cb_ls=1; _cb=BLfE4GBNSwJsLxiQ6; AMP_TOKEN=%24NOT_FOUND; __qca=P0-1753552556-1605535506773; mnet_session_depth=2%7C1605535504670; aasd=2%7C1605535505885; __pvi=%7B%22id%22%3A%22v-2020-11-16-15-05-06-660-0XYRWHdQ2XYGbaoP-46b9447a9edebf759b8530b76e3d77fb%22%2C%22domain%22%3A%22.forbes.com%22%2C%22time%22%3A1605535556929%7D; xbc=%7Bjzx%7DTSec02msUmzkbggY3uSB4x2gGnm1UBtgCEjruTUCsjodOn-ho2bBitYYAE8Wj1i1AsluUTc70wVCtGSzun5713YJ_vNU7uygAa3M3ny-XzzOp_CsCgfU3k8e_J8gocbcboz88mUS39sdnXYxb82rI1XecPrFioPfHcuQebQwPSy3ueG7x_j8wu6BCd9t0FTHUw-9eIvfWF1WWnXIvcJfs3MZ5QDIUBA9qEt2a7BQs_lhesgShrkv-PkBiPzR60tXNA8DkrlHZF_puETTlo-Xiw; _chartbeat2=.1594934728317.1605535561012.0000000000000001.BFC6ndInmcUBhHH5EC_OdRH7Niui.1; QSI_HistorySession=https%3A%2F%2Fwww.forbes.com%2Fglobal2000%2F%235197f5e1335d~1605525574002%7Chttps%3A%2F%2Fwww.forbes.com%2Fglobal2000%2F%234f485ec3335d~1605525601420%7Chttps%3A%2F%2Fwww.forbes.com%2Fglobal2000%2F%2367f91b9335d8~1605535512891%7Chttps%3A%2F%2Fwww.forbes.com%2Fglobal2000%2F%234c10589f335d~1605535561525' \
      --compressed
    

    解决方案是为请求提供一些额外的cookie。 使其工作的最小cookie集如下所示:

    curl 'https://www.forbes.com/forbesapi/org/global2000/2020/position/true.json?limit=2000'
    -H 'cookie: notice_behavior=expressed,eu; notice_gdpr_prefs=0,1,2:1a8b5228dd7ff0717196863a5d28ce6c;'   --compressed
    
    因为WebClient抱怨无效字符(:和,) 我最终得到了以下最小cookies集:

    @Bean
    public WebClient forbesWebClient() {
        return WebClient.builder()
                .baseUrl(BASE_URL)
                .defaultCookies(cookies -> {
                    cookies.add("notice_behavior", "expressed");
                    cookies.add("notice_gdpr_prefs", "0");
                })
                .codecs(configurer -> configurer.defaultCodecs().maxInMemorySize(MB5))
                .build();
    }