Send only 75 "tokens" to Ex Libris' Summon

[maxkadel]

Expected behavior

When a user puts in a very long query, it is still queried against the Summon article service

Actual behavior

Queries with many "tokens" (words and boolean operators combined) only get an error from Summon and are not searched.

example url with many tokens

produces

{
  "error": {
    "problem":"UPSTREAM_ERROR",
    "message":"Too many tokens in Query. Max tokens: 75 Token Count: 130"
  }
}

Steps to replicate

• Put a string with over 75 words in an allsearch-api search (start with https://allsearch-api.princeton.edu/search/article?query= then put over 75 words after)

Impact of this bug

Very long queries are not searched against the articles at all

Honeybadger link

https://app.honeybadger.io/projects/114176/faults/109358488/01JCH189R3HH5Z6ZDPZR0EWD4V?page=0

Implementation notes, if any

• Truncate queries to Summon to 75 "tokens"
• See Summon Large Search Queries documentation
• See similar ticket for long strings - Truncate queries to Ex Libris' Summon to 1000 characters #322