Skip to content
This repository has been archived by the owner on Oct 10, 2022. It is now read-only.

Travis issues #34

Open
stephlocke opened this issue Jan 18, 2019 · 11 comments
Open

Travis issues #34

stephlocke opened this issue Jan 18, 2019 · 11 comments
Labels
bug 🐛

Comments

@stephlocke
Copy link
Contributor

test_check("HIBPwned")
── 1. Failure: account_breaches works for a single account (@test-account_breach
res[[acct]] inherits from NULL not data.frame.
── 2. Error: account_breaches works for a single account (@test-account_breaches
is.numeric(act$val) is not TRUE
1: expect_gt(ncol(res[[acct]]), 1) at testthat/test-account_breaches.R:13
2: expect_compare(">", act, exp)
3: stopifnot(is.numeric(act$val), length(act$val) == 1)
4: stop(msg, call. = FALSE, domain = NA)
── 3. Failure: account_breaches works for multiple accounts (@test-account_breac
res[[a]] inherits from NULL not data.frame.
── 4. Error: account_breaches works for multiple accounts (@test-account_breache
is.numeric(act$val) is not TRUE
1: expect_gte(ncol(res[[a]]), 1) at testthat/test-account_breaches.R:59
2: expect_compare(">=", act, exp)
3: stopifnot(is.numeric(act$val), length(act$val) == 1)
4: stop(msg, call. = FALSE, domain = NA)
── 5. Error: account_breaches can return a single data.frame (@test-account_brea
subscript out of bounds
1: account_breaches("[email protected]", as_list = FALSE) at testthat/test-account_breaches.R:126
@maelle
Copy link
Contributor

maelle commented Jan 22, 2019

The user agent seems to be passed correctly. The request is however forbidden.

resp$status_http()
<Status code: 403>
  Message: Forbidden
  Explanation: Request forbidden -- authorization will not help

@maelle
Copy link
Contributor

maelle commented Jan 22, 2019

In the API docs the only reason given for a 403 error is a missing user agent. https://haveibeenpwned.com/API/v2

We passe the User Agent in a list e.g.

  list(`api-version` = "2"
       , `User-Agent` = agent)
#> Error in eval(expr, envir, enclos): object 'agent' not found

Created on 2019-01-22 by the reprex package (v0.2.1)

And then this is how we pass it to crul, as headers

 crul::HttpClient$new(url = url,
                       headers = headers)$get()

@maelle
Copy link
Contributor

maelle commented Jan 22, 2019

Oh but now the Travis build no longer fails?!

@maelle
Copy link
Contributor

maelle commented Jan 22, 2019

Locally I still get the failures. 🤔

@maelle
Copy link
Contributor

maelle commented Jan 28, 2019

@stephlocke we can rule out crul being the issue

account <- "[email protected]"
encoded <- urltools::url_encode(account)
URL <- paste0( # nolint
  "https://haveibeenpwned.com/api/v2/breachedaccount/"
  , encoded
)

headers <- list(`api-version` = "2", 
                `User-Agent` = "R package HIBPwned")
res <- crul::HttpClient$new(url = URL,
                            headers = headers)$get()
res$request_headers
#> $`Accept-Encoding`
#> [1] "gzip, deflate"
#> 
#> $Accept
#> [1] "application/json, text/xml, application/xml, */*"
#> 
#> $`api-version`
#> [1] "2"
#> 
#> $`User-Agent`
#> [1] "R package HIBPwned"

httr::GET(URL, 
         httr::user_agent("R package HIBPwned"))
#> Response [https://haveibeenpwned.com/api/v2/breachedaccount/maelle.salmon%40yahoo.se]
#>   Date: 2019-01-28 15:59
#>   Status: 403
#>   Content-Type: text/html; charset=UTF-8
#>   Size: 1.96 kB
#> <!DOCTYPE html>
#> <head>
#> <title>Request Blocked</title>
#> <meta charset="UTF-8" />
#> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
#> </head>
#> <body>
#> <h1>You have been blocked from accessing this resource on Have I Been Pw...
#> 
#> <p>This may be due to violating one or more of <a href="https://haveibee...
#> ...

Created on 2019-01-28 by the reprex package (v0.2.1)

Session info 
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.4.4 (2018-03-15)
#>  os       Ubuntu 18.04.1 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_US                       
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       Europe/Paris                
#>  date     2019-01-28                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version    date       lib source                         
#>  assertthat    0.2.0      2017-04-11 [1] CRAN (R 3.4.4)                 
#>  backports     1.1.3      2018-12-14 [1] CRAN (R 3.4.4)                 
#>  callr         3.1.1      2018-12-21 [1] CRAN (R 3.4.4)                 
#>  cli           1.0.1      2018-09-25 [1] CRAN (R 3.4.4)                 
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 3.4.4)                 
#>  crul          0.7.0      2019-01-04 [1] CRAN (R 3.4.4)                 
#>  curl          3.3        2019-01-10 [1] CRAN (R 3.4.4)                 
#>  desc          1.2.0      2018-05-01 [1] CRAN (R 3.4.4)                 
#>  devtools      2.0.1      2018-10-26 [1] CRAN (R 3.4.4)                 
#>  digest        0.6.18     2018-10-10 [1] CRAN (R 3.4.4)                 
#>  evaluate      0.12       2018-10-09 [1] CRAN (R 3.4.4)                 
#>  fs            1.2.6      2018-08-23 [1] CRAN (R 3.4.4)                 
#>  glue          1.3.0.9000 2019-01-22 [1] Github (tidyverse/glue@0549457)
#>  highr         0.7        2018-06-09 [1] CRAN (R 3.4.4)                 
#>  htmltools     0.3.6      2017-04-28 [1] CRAN (R 3.4.4)                 
#>  httpcode      0.2.0      2016-11-14 [1] CRAN (R 3.4.4)                 
#>  httr          1.4.0      2018-12-11 [1] CRAN (R 3.4.4)                 
#>  knitr         1.21       2018-12-10 [1] CRAN (R 3.4.4)                 
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 3.4.4)                 
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 3.4.4)                 
#>  pkgbuild      1.0.2      2018-10-16 [1] CRAN (R 3.4.4)                 
#>  pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.4.4)                 
#>  prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.4.4)                 
#>  processx      3.2.1      2018-12-05 [1] CRAN (R 3.4.4)                 
#>  ps            1.3.0      2018-12-21 [1] CRAN (R 3.4.4)                 
#>  R6            2.3.0      2018-10-04 [1] CRAN (R 3.4.4)                 
#>  Rcpp          1.0.0      2018-11-07 [1] CRAN (R 3.4.4)                 
#>  remotes       2.0.2      2018-10-30 [1] CRAN (R 3.4.4)                 
#>  rlang         0.3.1      2019-01-08 [1] CRAN (R 3.4.4)                 
#>  rmarkdown     1.11       2018-12-08 [1] CRAN (R 3.4.4)                 
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.4.4)                 
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.4.4)                 
#>  stringi       1.2.4      2018-07-20 [1] CRAN (R 3.4.4)                 
#>  stringr       1.3.1      2018-05-10 [1] CRAN (R 3.4.4)                 
#>  testthat      2.0.1      2018-10-13 [1] CRAN (R 3.4.4)                 
#>  triebeard     0.3.0      2016-08-04 [1] CRAN (R 3.4.4)                 
#>  urltools      1.7.1      2018-08-03 [1] CRAN (R 3.4.4)                 
#>  usethis       1.4.0.9000 2019-01-03 [1] Github (r-lib/usethis@85bf30a) 
#>  withr         2.1.2      2018-03-15 [1] CRAN (R 3.4.4)                 
#>  xfun          0.4        2018-10-23 [1] CRAN (R 3.4.4)                 
#>  yaml          2.2.0      2018-07-25 [1] CRAN (R 3.4.4)                 
#> 
#> [1] /home/maelle/R/x86_64-pc-linux-gnu-library/3.4
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library

@maelle
Copy link
Contributor

maelle commented Jan 28, 2019

Hypotheses:

  • email address used in examples looks spammy because used too much (idea of @stephlocke's). In that case wouldn't we get a different code if searching an email address for the first time?
  • user agent looking spammy?! I tried changing it, no success.
  • IP address?

@maelle
Copy link
Contributor

maelle commented Jan 28, 2019

Note that the build passes on Travis.

@maelle
Copy link
Contributor

maelle commented Jan 28, 2019

Possibly related michenriksen/hibp#3

@maelle
Copy link
Contributor

maelle commented Jan 28, 2019

See also https://community.home-assistant.io/t/haveibeenpwned-stopped-working-failed-fetching-data-http-status-code-403/92680/4

@maelle
Copy link
Contributor

maelle commented Apr 3, 2019

In the thread above folks write that they had to contact Troy Hunt to get their IP unblocked. 🤔

@stephlocke if CRAN's IP was blocked by our tests, it's quite bad for #35

Besides, if we contacted the API maintainer, I wonder whether we could ask whether there was a reason in all the 403 errors/IP blocking. Bad luck for us or did we do something wrong?

@maelle
Copy link
Contributor

maelle commented Apr 3, 2019

For info now the tests pass on Travis. 🤷‍♀️

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug 🐛
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@stephlocke @maelle