diff --git a/docs/docs/how_to/tools_builtin.ipynb b/docs/docs/how_to/tools_builtin.ipynb index fbdc1636454a8..7d4f05a7d4ab2 100644 --- a/docs/docs/how_to/tools_builtin.ipynb +++ b/docs/docs/how_to/tools_builtin.ipynb @@ -36,7 +36,7 @@ "\n", "When using 3rd party tools, make sure that you understand how the tool works, what permissions\n", "it has. Read over its documentation and check if anything is required from you\n", - "from a security point of view. Please see our [security](https://python.langchain.com/v0.1/docs/security/) \n", + "from a security point of view. Please see our [security](https://python.langchain.com/v0.2/docs/security/) \n", "guidelines for more information.\n", "\n", ":::\n", diff --git a/docs/docs/integrations/document_loaders/recursive_url.ipynb b/docs/docs/integrations/document_loaders/recursive_url.ipynb index 3573869cec53a..fd86ba683261b 100644 --- a/docs/docs/integrations/document_loaders/recursive_url.ipynb +++ b/docs/docs/integrations/document_loaders/recursive_url.ipynb @@ -7,152 +7,309 @@ "source": [ "# Recursive URL\n", "\n", - "We may want to process load all URLs under a root directory.\n", - "\n", - "For example, let's look at the [Python 3.9 Document](https://docs.python.org/3.9/).\n", - "\n", - "This has many interesting child pages that we may want to read in bulk.\n", - "\n", - "Of course, the `WebBaseLoader` can load a list of pages. \n", - "\n", - "But, the challenge is traversing the tree of child pages and actually assembling that list!\n", - " \n", - "We do this using the `RecursiveUrlLoader`.\n", - "\n", - "This also gives us the flexibility to exclude some children, customize the extractor, and more." + "The `RecursiveUrlLoader` lets you recursively scrape all child links from a root URL and parse them into Documents." ] }, { "cell_type": "markdown", - "id": "1be8094f", + "id": "947d29e7-3679-483d-973f-79ea3403a370", "metadata": {}, "source": [ - "# Parameters\n", - "- url: str, the target url to crawl.\n", - "- exclude_dirs: Optional[str], webpage directories to exclude.\n", - "- use_async: Optional[bool], wether to use async requests, using async requests is usually faster in large tasks. However, async will disable the lazy loading feature(the function still works, but it is not lazy). By default, it is set to False.\n", - "- extractor: Optional[Callable[[str], str]], a function to extract the text of the document from the webpage, by default it returns the page as it is. It is recommended to use tools like goose3 and beautifulsoup to extract the text. By default, it just returns the page as it is.\n", - "- max_depth: Optional[int] = None, the maximum depth to crawl. By default, it is set to 2. If you need to crawl the whole website, set it to a number that is large enough would simply do the job.\n", - "- timeout: Optional[int] = None, the timeout for each request, in the unit of seconds. By default, it is set to 10.\n", - "- prevent_outside: Optional[bool] = None, whether to prevent crawling outside the root url. By default, it is set to True." + "## Setup\n", + "\n", + "The `RecursiveUrlLoader` lives in the `langchain-community` package. There's no other required packages, though you will get richer default Document metadata if you have ``beautifulsoup4` installed as well." ] }, { "cell_type": "code", "execution_count": null, - "id": "23c18539", + "id": "23359ab0-8056-4dee-8bff-c38dc079f17f", "metadata": {}, "outputs": [], "source": [ - "from langchain_community.document_loaders.recursive_url_loader import RecursiveUrlLoader" + "%pip install -qU langchain-community beautifulsoup4" ] }, { "cell_type": "markdown", - "id": "6384c057", + "id": "07985766-e4e9-4ea1-8a18-924fa4f294e5", "metadata": {}, "source": [ - "Let's try a simple example." + "## Instantiation\n", + "\n", + "Now we can instantiate our document loader object and load Documents:" ] }, { "cell_type": "code", - "execution_count": null, - "id": "55394afe", + "execution_count": 1, + "id": "cb208dcf-9ce9-4197-bc44-b80d20aa4e50", "metadata": {}, "outputs": [], "source": [ - "from bs4 import BeautifulSoup as Soup\n", + "from langchain_community.document_loaders import RecursiveUrlLoader\n", "\n", - "url = \"https://docs.python.org/3.9/\"\n", "loader = RecursiveUrlLoader(\n", - " url=url, max_depth=2, extractor=lambda x: Soup(x, \"html.parser\").text\n", - ")\n", - "docs = loader.load()" + " \"https://docs.python.org/3.9/\",\n", + " # max_depth=2,\n", + " # use_async=False,\n", + " # extractor=None,\n", + " # metadata_extractor=None,\n", + " # exclude_dirs=(),\n", + " # timeout=10,\n", + " # check_response_status=True,\n", + " # continue_on_failure=True,\n", + " # prevent_outside=True,\n", + " # base_url=None,\n", + " # ...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0fac4425-735f-487d-a12b-c8ed2a209039", + "metadata": {}, + "source": [ + "## Load\n", + "\n", + "Use ``.load()`` to synchronously load into memory all Documents, with one\n", + "Document per visited URL. Starting from the initial URL, we recurse through\n", + "all linked URLs up to the specified max_depth.\n", + "\n", + "Let's run through a basic example of how to use the `RecursiveUrlLoader` on the [Python 3.9 Documentation](https://docs.python.org/3.9/)." ] }, { "cell_type": "code", - "execution_count": 3, - "id": "084fb2ce", + "execution_count": 2, + "id": "a30843c8-4a59-43dc-bf60-f26532f0f8e1", "metadata": {}, "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/bagatur/.pyenv/versions/3.9.1/lib/python3.9/html/parser.py:170: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features=\"xml\"` into the BeautifulSoup constructor.\n", + " k = self.parse_starttag(i)\n" + ] + }, { "data": { "text/plain": [ - "'\\n\\n\\n\\n\\nPython Frequently Asked Questions — Python 3.'" + "{'source': 'https://docs.python.org/3.9/',\n", + " 'content_type': 'text/html',\n", + " 'title': '3.9.19 Documentation',\n", + " 'language': None}" ] }, - "execution_count": 3, + "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "docs[0].page_content[:50]" + "docs = loader.load()\n", + "docs[0].metadata" + ] + }, + { + "cell_type": "markdown", + "id": "211856ed-6dd7-46c6-859e-11aaea9093db", + "metadata": {}, + "source": [ + "Great! The first document looks like the root page we started from. Let's look at the metadata of the next document" ] }, { "cell_type": "code", - "execution_count": 4, - "id": "13bd7e16", + "execution_count": 3, + "id": "2d842c03-fab8-4097-9f4f-809b2e71c0ba", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'source': 'https://docs.python.org/3.9/library/index.html',\n", - " 'title': 'The Python Standard Library — Python 3.9.17 documentation',\n", + "{'source': 'https://docs.python.org/3.9/using/index.html',\n", + " 'content_type': 'text/html',\n", + " 'title': 'Python Setup and Usage — Python 3.9.19 documentation',\n", " 'language': None}" ] }, - "execution_count": 4, + "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "docs[-1].metadata" + "docs[1].metadata" ] }, { "cell_type": "markdown", - "id": "5866e5a6", + "id": "f5714ace-7cc5-4c5c-9426-f68342880da0", "metadata": {}, "source": [ - "However, since it's hard to perform a perfect filter, you may still see some irrelevant results in the results. You can perform a filter on the returned documents by yourself, if it's needed. Most of the time, the returned results are good enough." + "That url looks like a child of our root page, which is great! Let's move on from metadata to examine the content of one of our documents" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "51dc6c67-6857-4298-9472-08b147f3a631", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\n", + "\n", + " \n", + " 3.9.19 Documentation\n", + " \n", + " \n", + " str:\n", + " soup = BeautifulSoup(html, \"lxml\")\n", + " return re.sub(r\"\\n\\n+\", \"\\n\\n\", soup.text).strip()\n", + "\n", + "\n", + "loader = RecursiveUrlLoader(\"https://docs.python.org/3.9/\", extractor=bs4_extractor)\n", "docs = loader.load()\n", - "len(docs)" + "print(docs[0].page_content[:200])" + ] + }, + { + "cell_type": "markdown", + "id": "c8e8a826", + "metadata": {}, + "source": [ + "This looks much nicer!\n", + "\n", + "You can similarly pass in a `metadata_extractor` to customize how Document metadata is extracted from the HTTP response. See the [API reference](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.recursive_url_loader.RecursiveUrlLoader.html) for more on this." + ] + }, + { + "cell_type": "markdown", + "id": "1dddbc94", + "metadata": {}, + "source": [ + "## Lazy loading\n", + "\n", + "If we're loading a large number of Documents and our downstream operations can be done over subsets of all loaded Documents, we can lazily load our Documents one at a time to minimize our memory footprint:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "7d0114fc", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/var/folders/4j/2rz3865x6qg07tx43146py8h0000gn/T/ipykernel_73962/2110507528.py:6: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features=\"xml\"` into the BeautifulSoup constructor.\n", + " soup = BeautifulSoup(html, \"lxml\")\n" + ] + } + ], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "id": "f88a7c2f-35df-4c3a-b238-f91be2674b96", + "metadata": {}, + "source": [ + "In this example we never have more than 10 Documents loaded into memory at a time." + ] + }, + { + "cell_type": "markdown", + "id": "3e4d1c8f", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "These examples show just a few of the ways in which you can modify the default `RecursiveUrlLoader`, but there are many more modifications that can be made to best fit your use case. Using the parameters `link_regex` and `exclude_dirs` can help you filter out unwanted URLs, `aload()` and `alazy_load()` can be used for aynchronous loading, and more.\n", + "\n", + "For detailed information on configuring and calling the ``RecursiveUrlLoader``, please see the API reference: https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.recursive_url_loader.RecursiveUrlLoader.html." ] } ], @@ -172,7 +329,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.12" + "version": "3.11.9" } }, "nbformat": 4, diff --git a/libs/community/Makefile b/libs/community/Makefile index e7b1b6465a15e..bb5631a2ec663 100644 --- a/libs/community/Makefile +++ b/libs/community/Makefile @@ -44,6 +44,7 @@ lint_tests: MYPY_CACHE=.mypy_cache_test lint lint_diff lint_package lint_tests: ./scripts/check_pydantic.sh . ./scripts/lint_imports.sh + ./scripts/check_pickle.sh . poetry run ruff . [ "$(PYTHON_FILES)" = "" ] || poetry run ruff format $(PYTHON_FILES) --diff [ "$(PYTHON_FILES)" = "" ] || poetry run ruff --select I $(PYTHON_FILES) diff --git a/libs/community/langchain_community/document_loaders/recursive_url_loader.py b/libs/community/langchain_community/document_loaders/recursive_url_loader.py index 83e62e1d8963f..62e7352e3d573 100644 --- a/libs/community/langchain_community/document_loaders/recursive_url_loader.py +++ b/libs/community/langchain_community/document_loaders/recursive_url_loader.py @@ -37,7 +37,7 @@ def _metadata_extractor( except ImportError: logger.warning( "The bs4 package is required for default metadata extraction. " - "Please install it with `pip install bs4`." + "Please install it with `pip install -U beautifulsoup4`." ) return metadata soup = BeautifulSoup(raw_html, "html.parser") @@ -51,7 +51,7 @@ def _metadata_extractor( class RecursiveUrlLoader(BaseLoader): - """Load all child links from a URL page. + """Recursively load all child links from a root URL. **Security Note**: This loader is a crawler that will start crawling at a given URL and then expand to crawl child links recursively. @@ -79,8 +79,196 @@ class RecursiveUrlLoader(BaseLoader): GET request to an endpoint on Bob's site. Both sites are hosted on the same host, so such a request would not be prevented by default. - See https://python.langchain.com/docs/security - """ + See https://python.langchain.com/v0.2/docs/security/ + + Setup: + + This class has no required additional dependencies. You can optionally install + ``beautifulsoup4`` for richer default metadata extraction: + + .. code-block:: bash + + pip install -U beautifulsoup4 + + Instantiate: + .. code-block:: python + + from langchain_community.document_loaders import RecursiveUrlLoader + + loader = RecursiveUrlLoader( + "https://docs.python.org/3.9/", + # max_depth=2, + # use_async=False, + # extractor=None, + # metadata_extractor=None, + # exclude_dirs=(), + # timeout=10, + # check_response_status=True, + # continue_on_failure=True, + # prevent_outside=True, + # base_url=None, + # ... + ) + + Load: + Use ``.load()`` to synchronously load into memory all Documents, with one + Document per visited URL. Starting from the initial URL, we recurse through + all linked URLs up to the specified max_depth. + + .. code-block:: python + + docs = loader.load() + print(docs[0].page_content[:100]) + print(docs[0].metadata) + + .. code-block:: python + + + + + + < + {'source': 'https://docs.python.org/3.9/', 'content_type': 'text/html', 'title': '3.9.19 Documentation', 'language': None} + + Async load: + .. code-block:: python + + docs = await loader.aload() + print(docs[0].page_content[:100]) + print(docs[0].metadata) + + .. code-block:: python + + + + + + < + {'source': 'https://docs.python.org/3.9/', 'content_type': 'text/html', 'title': '3.9.19 Documentation', 'language': None} + + Lazy load: + .. code-block:: python + + docs = [] + docs_lazy = loader.lazy_load() + + # async variant: + # docs_lazy = await loader.alazy_load() + + for doc in docs_lazy: + docs.append(doc) + print(docs[0].page_content[:100]) + print(docs[0].metadata) + + .. code-block:: python + + + + + + < + {'source': 'https://docs.python.org/3.9/', 'content_type': 'text/html', 'title': '3.9.19 Documentation', 'language': None} + + Content parsing / extraction: + By default the loader sets the raw HTML from each link as the Document page + content. To parse this HTML into a more human/LLM-friendly format you can pass + in a custom ``extractor`` method: + + .. code-block:: python + + # This example uses `beautifulsoup4` and `lxml` + import re + from bs4 import BeautifulSoup + + def bs4_extractor(html: str) -> str: + soup = BeautifulSoup(html, "lxml") + return re.sub(r"\n\n+", "\n\n", soup.text).strip() + + loader = RecursiveUrlLoader( + "https://docs.python.org/3.9/", + extractor=bs4_extractor, + ) + print(loader.load()[0].page_content[:200]) + + + .. code-block:: python + + 3.9.19 Documentation + + Download + Download these documents + Docs by version + + Python 3.13 (in development) + Python 3.12 (stable) + Python 3.11 (security-fixes) + Python 3.10 (security-fixes) + Python 3.9 (securit + + Metadata extraction: + Similarly to content extraction, you can specify a metadata extraction function + to customize how Document metadata is extracted from the HTTP response. + + .. code-block:: python + + import aiohttp + import requests + from typing import Union + + def simple_metadata_extractor( + raw_html: str, url: str, response: Union[requests.Response, aiohttp.ClientResponse] + ) -> dict: + content_type = getattr(response, "headers").get("Content-Type", "") + return {"source": url, "content_type": content_type} + + loader = RecursiveUrlLoader( + "https://docs.python.org/3.9/", + metadata_extractor=simple_metadata_extractor, + ) + loader.load()[0].metadata + + .. code-block:: python + + {'source': 'https://docs.python.org/3.9/', 'content_type': 'text/html'} + + Filtering URLs: + You may not always want to pull every URL from a website. There are four parameters + that allow us to control what URLs we pull recursively. First, we can set the + ``prevent_outside`` parameter to prevent URLs outside of the ``base_url`` from + being pulled. Note that the ``base_url`` does not need to be the same as the URL we + pass in, as shown below. We can also use ``link_regex`` and ``exclude_dirs`` to be + more specific with the URLs that we select. In this example, we only pull websites + from the python docs, which contain the string "index" somewhere and are not + located in the FAQ section of the website. + + .. code-block:: python + + loader = RecursiveUrlLoader( + "https://docs.python.org/3.9/", + prevent_outside=True, + base_url="https://docs.python.org", + link_regex=r']*?\s+)?href="([^"]*(?=index)[^"]*)"', + exclude_dirs=['https://docs.python.org/3.9/faq'] + ) + docs = loader.load() + + .. code-block:: python + + ['https://docs.python.org/3.9/', + 'https://docs.python.org/3.9/py-modindex.html', + 'https://docs.python.org/3.9/genindex.html', + 'https://docs.python.org/3.9/tutorial/index.html', + 'https://docs.python.org/3.9/using/index.html', + 'https://docs.python.org/3.9/extending/index.html', + 'https://docs.python.org/3.9/installing/index.html', + 'https://docs.python.org/3.9/library/index.html', + 'https://docs.python.org/3.9/c-api/index.html', + 'https://docs.python.org/3.9/howto/index.html', + 'https://docs.python.org/3.9/distributing/index.html', + 'https://docs.python.org/3.9/reference/index.html', + 'https://docs.python.org/3.9/whatsnew/index.html'] + + """ # noqa: E501 def __init__( self, @@ -107,12 +295,12 @@ def __init__( url: The URL to crawl. max_depth: The max depth of the recursive loading. use_async: Whether to use asynchronous loading. - If True, this function will not be lazy, but it will still work in the + If True, lazy_load function will not be lazy, but it will still work in the expected way, just not lazy. - extractor: A function to extract document contents from raw html. + extractor: A function to extract document contents from raw HTML. When extract function returns an empty string, the document is - ignored. - metadata_extractor: A function to extract metadata from args: raw html, the + ignored. Default returns the raw HTML. + metadata_extractor: A function to extract metadata from args: raw HTML, the source url, and the requests.Response/aiohttp.ClientResponse object (args in that order). Default extractor will attempt to use BeautifulSoup4 to extract the @@ -254,13 +442,6 @@ async def _async_get_child_links_recursive( "Async functions forbidden when not initialized with `use_async`" ) - try: - import aiohttp - except ImportError: - raise ImportError( - "The aiohttp package is required for the RecursiveUrlLoader. " - "Please install it with `pip install aiohttp`." - ) if depth >= self.max_depth: return [] diff --git a/libs/community/langchain_community/llms/databricks.py b/libs/community/langchain_community/llms/databricks.py index 8da2f1db8382a..e350a173c8f8e 100644 --- a/libs/community/langchain_community/llms/databricks.py +++ b/libs/community/langchain_community/llms/databricks.py @@ -242,7 +242,7 @@ def _load_pickled_fn_from_hex_string( raise ValueError(f"Please install cloudpickle>=2.0.0. Error: {e}") try: - return cloudpickle.loads(bytes.fromhex(data)) + return cloudpickle.loads(bytes.fromhex(data)) # ignore[pickle]: explicit-opt-in except Exception as e: raise ValueError( f"Failed to load the pickled function from a hexadecimal string. Error: {e}" diff --git a/libs/community/langchain_community/llms/self_hosted.py b/libs/community/langchain_community/llms/self_hosted.py index e94f2d405417c..d1749f43ac02f 100644 --- a/libs/community/langchain_community/llms/self_hosted.py +++ b/libs/community/langchain_community/llms/self_hosted.py @@ -36,7 +36,9 @@ def _send_pipeline_to_device(pipeline: Any, device: int) -> Any: """Send a pipeline to a device on the cluster.""" if isinstance(pipeline, str): with open(pipeline, "rb") as f: - pipeline = pickle.load(f) + # This code path can only be triggered if the user + # passed allow_dangerous_deserialization=True + pipeline = pickle.load(f) # ignore[pickle]: explicit-opt-in if importlib.util.find_spec("torch") is not None: import torch diff --git a/libs/community/langchain_community/retrievers/tfidf.py b/libs/community/langchain_community/retrievers/tfidf.py index e7cfc7b9eef34..dd19b796c1aba 100644 --- a/libs/community/langchain_community/retrievers/tfidf.py +++ b/libs/community/langchain_community/retrievers/tfidf.py @@ -152,6 +152,8 @@ def load_local( # Load docs and tfidf array as pickle. with open(path / f"{file_name}.pkl", "rb") as f: - docs, tfidf_array = pickle.load(f) + # This code path can only be triggered if the user + # passed allow_dangerous_deserialization=True + docs, tfidf_array = pickle.load(f) # ignore[pickle]: explicit-opt-in return cls(vectorizer=vectorizer, docs=docs, tfidf_array=tfidf_array) diff --git a/libs/community/langchain_community/tools/tavily_search/tool.py b/libs/community/langchain_community/tools/tavily_search/tool.py index 1d05387927087..e93b98695cdae 100644 --- a/libs/community/langchain_community/tools/tavily_search/tool.py +++ b/libs/community/langchain_community/tools/tavily_search/tool.py @@ -19,7 +19,51 @@ class TavilyInput(BaseModel): class TavilySearchResults(BaseTool): - """Tool that queries the Tavily Search API and gets back json.""" + """Tool that queries the Tavily Search API and gets back json. + + Setup: + Install ``langchain-openai`` and ``tavily-python``, and set environment variable ``TAVILY_API_KEY``. + + .. code-block:: bash + + pip install -U langchain-openai + export TAVILY_API_KEY="your-api-key" + + Instantiate: + + .. code-block:: python + + from langchain_community.tools.tavily_search import TavilySearchResults + + tool = TavilySearchResults( + # max_results= 5 + # search_depth = "advanced" + # include_domains = [] + # exclude_domains = [] + # include_answer = False + # include_raw_content = False + # include_images = False + ) + + Invoke: + + .. code-block:: python + + tool = TavilySearchResults(max_results=3) + tool.invoke("What is the weather?") + + .. code-block:: python + + [{'url': 'https://www.weatherapi.com/', + 'content': "{'location': {'name': 'Current', 'region': 'Harbour Island', 'country': 'Bahamas', 'lat': 25.43, 'lon': -76.78, 'tz_id': 'America/Nassau', 'localtime_epoch': 1718077801, 'localtime': '2024-06-10 23:50'}, 'current': {'last_updated_epoch': 1718077500, 'last_updated': '2024-06-10 23:45', 'temp_c': 27.9, 'temp_f': 82.1, 'is_day': 0, 'condition': {'text': 'Patchy rain nearby', 'icon': '//cdn.weatherapi.com/weather/64x64/night/176.png', 'code': 1063}, 'wind_mph': 14.5, 'wind_kph': 23.4, 'wind_degree': 161, 'wind_dir': 'SSE', 'pressure_mb': 1014.0, 'pressure_in': 29.94, 'precip_mm': 0.01, 'precip_in': 0.0, 'humidity': 88, 'cloud': 74, 'feelslike_c': 33.1, 'feelslike_f': 91.5, 'windchill_c': 27.9, 'windchill_f': 82.1, 'heatindex_c': 33.1, 'heatindex_f': 91.5, 'dewpoint_c': 25.6, 'dewpoint_f': 78.1, 'vis_km': 10.0, 'vis_miles': 6.0, 'uv': 1.0, 'gust_mph': 20.4, 'gust_kph': 32.9}}"}, + {'url': 'https://www.localconditions.com/weather-ninnescah-kansas/67069/', + 'content': 'The following chart reports what the hourly Ninnescah, KS temperature has been today, from 12:56 AM to 3:56 AM Tue, May 21st 2024. The lowest temperature reading has been 73.04 degrees fahrenheit at 3:56 AM, while the highest temperature is 75.92 degrees fahrenheit at 12:56 AM. Ninnescah KS detailed current weather report for 67069 in Kansas.'}, + {'url': 'https://www.weather.gov/forecastmaps/', + 'content': 'Short Range Forecasts. Short range forecast products depicting pressure patterns, circulation centers and fronts, and types and extent of precipitation. 12 Hour | 24 Hour | 36 Hour | 48 Hour.'}] + + When converting ``TavilySearchResults`` to a tool, you may want to not return all of the content resulting from ``invoke``. You can select what parts of the response to keep depending on your use case. + + """ # noqa: E501 name: str = "tavily_search_results_json" description: str = ( @@ -29,6 +73,19 @@ class TavilySearchResults(BaseTool): ) api_wrapper: TavilySearchAPIWrapper = Field(default_factory=TavilySearchAPIWrapper) # type: ignore[arg-type] max_results: int = 5 + """Max search results to return, default is 5""" + search_depth: str = "advanced" + '''The depth of the search. It can be "basic" or "advanced"''' + include_domains: List[str] = [] + """A list of domains to specifically include in the search results. Default is None, which includes all domains.""" # noqa: E501 + exclude_domains: List[str] = [] + """A list of domains to specifically exclude from the search results. Default is None, which doesn't exclude any domains.""" # noqa: E501 + include_answer: bool = False + """Include a short answer to original query in the search results. Default is False.""" # noqa: E501 + include_raw_content: bool = False + """Include cleaned and parsed HTML of each site search results. Default is False.""" + include_images: bool = False + """Include a list of query related images in the response. Default is False.""" args_schema: Type[BaseModel] = TavilyInput def _run( @@ -41,6 +98,12 @@ def _run( return self.api_wrapper.results( query, self.max_results, + self.search_depth, + self.include_domains, + self.exclude_domains, + self.include_answer, + self.include_raw_content, + self.include_images, ) except Exception as e: return repr(e) @@ -55,6 +118,12 @@ async def _arun( return await self.api_wrapper.results_async( query, self.max_results, + self.search_depth, + self.include_domains, + self.exclude_domains, + self.include_answer, + self.include_raw_content, + self.include_images, ) except Exception as e: return repr(e) diff --git a/libs/community/langchain_community/vectorstores/annoy.py b/libs/community/langchain_community/vectorstores/annoy.py index 3d749c927e637..59058b7265f42 100644 --- a/libs/community/langchain_community/vectorstores/annoy.py +++ b/libs/community/langchain_community/vectorstores/annoy.py @@ -456,7 +456,14 @@ def load_local( annoy = guard_import("annoy") # load docstore and index_to_docstore_id with open(path / "index.pkl", "rb") as file: - docstore, index_to_docstore_id, config_object = pickle.load(file) + # Code path can only be reached if allow_dangerous_deserialization is True + ( + docstore, + index_to_docstore_id, + config_object, + ) = pickle.load( # ignore[pickle]: explicit-opt-in + file + ) f = int(config_object["ANNOY"]["f"]) metric = config_object["ANNOY"]["metric"] diff --git a/libs/community/langchain_community/vectorstores/faiss.py b/libs/community/langchain_community/vectorstores/faiss.py index ae0f45f7e0774..5e907a795f504 100644 --- a/libs/community/langchain_community/vectorstores/faiss.py +++ b/libs/community/langchain_community/vectorstores/faiss.py @@ -1093,7 +1093,13 @@ def load_local( # load docstore and index_to_docstore_id with open(path / f"{index_name}.pkl", "rb") as f: - docstore, index_to_docstore_id = pickle.load(f) + ( + docstore, + index_to_docstore_id, + ) = pickle.load( # ignore[pickle]: explicit-opt-in + f + ) + return cls(embeddings, index, docstore, index_to_docstore_id, **kwargs) def serialize_to_bytes(self) -> bytes: @@ -1105,10 +1111,31 @@ def deserialize_from_bytes( cls, serialized: bytes, embeddings: Embeddings, + *, + allow_dangerous_deserialization: bool = False, **kwargs: Any, ) -> FAISS: """Deserialize FAISS index, docstore, and index_to_docstore_id from bytes.""" - index, docstore, index_to_docstore_id = pickle.loads(serialized) + if not allow_dangerous_deserialization: + raise ValueError( + "The de-serialization relies loading a pickle file. " + "Pickle files can be modified to deliver a malicious payload that " + "results in execution of arbitrary code on your machine." + "You will need to set `allow_dangerous_deserialization` to `True` to " + "enable deserialization. If you do this, make sure that you " + "trust the source of the data. For example, if you are loading a " + "file that you created, and know that no one else has modified the " + "file, then this is safe to do. Do not set this to `True` if you are " + "loading a file from an untrusted source (e.g., some random site on " + "the internet.)." + ) + ( + index, + docstore, + index_to_docstore_id, + ) = pickle.loads( # ignore[pickle]: explicit-opt-in + serialized + ) return cls(embeddings, index, docstore, index_to_docstore_id, **kwargs) def _select_relevance_score_fn(self) -> Callable[[float], float]: diff --git a/libs/community/langchain_community/vectorstores/scann.py b/libs/community/langchain_community/vectorstores/scann.py index 3824f2a7338cf..917a946cad86a 100644 --- a/libs/community/langchain_community/vectorstores/scann.py +++ b/libs/community/langchain_community/vectorstores/scann.py @@ -493,7 +493,13 @@ def load_local( # load docstore and index_to_docstore_id with open(path / "{index_name}.pkl".format(index_name=index_name), "rb") as f: - docstore, index_to_docstore_id = pickle.load(f) + ( + docstore, + index_to_docstore_id, + ) = pickle.load( # ignore[pickle]: explicit-opt-in + f + ) + return cls(embedding, index, docstore, index_to_docstore_id, **kwargs) def _select_relevance_score_fn(self) -> Callable[[float], float]: diff --git a/libs/community/langchain_community/vectorstores/tiledb.py b/libs/community/langchain_community/vectorstores/tiledb.py index d8c3c64087131..5db48a0786b77 100644 --- a/libs/community/langchain_community/vectorstores/tiledb.py +++ b/libs/community/langchain_community/vectorstores/tiledb.py @@ -188,7 +188,7 @@ def process_index_results( pickled_metadata = doc.get("metadata") result_doc = Document(page_content=str(doc["text"][0])) if pickled_metadata is not None: - metadata = pickle.loads( + metadata = pickle.loads( # ignore[pickle]: explicit-opt-in np.array(pickled_metadata.tolist()).astype(np.uint8).tobytes() ) result_doc.metadata = metadata diff --git a/libs/community/scripts/check_pickle.sh b/libs/community/scripts/check_pickle.sh new file mode 100755 index 0000000000000..036ff406173d3 --- /dev/null +++ b/libs/community/scripts/check_pickle.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# +# This checks for usage of pickle in the package. +# +# Usage: ./scripts/check_pickle.sh /path/to/repository +# +# Check if a path argument is provided +if [ $# -ne 1 ]; then + echo "Usage: $0 /path/to/repository" + exit 1 +fi + +repository_path="$1" + +# Search for lines matching the pattern within the specified repository +result=$(git -C "$repository_path" grep -E 'pickle.load\(|pickle.loads\(' | grep -v '# ignore\[pickle\]: explicit-opt-in') + +# Check if any matching lines were found +if [ -n "$result" ]; then + echo "ERROR: The following lines need to be updated:" + echo "$result" + echo "Please avoid using pickle or cloudpickle." + echo "If you must, then add:" + echo "1. A security notice (scan the code for examples)" + echo "2. Code path should be opt-in." + exit 1 +fi diff --git a/libs/experimental/langchain_experimental/agents/agent_toolkits/pandas/base.py b/libs/experimental/langchain_experimental/agents/agent_toolkits/pandas/base.py index 6fe319fa3e8da..ddf389f5e7d14 100644 --- a/libs/experimental/langchain_experimental/agents/agent_toolkits/pandas/base.py +++ b/libs/experimental/langchain_experimental/agents/agent_toolkits/pandas/base.py @@ -168,10 +168,23 @@ def create_pandas_dataframe_agent( number_of_head_rows: int = 5, extra_tools: Sequence[BaseTool] = (), engine: Literal["pandas", "modin"] = "pandas", + allow_dangerous_code: bool = False, **kwargs: Any, ) -> AgentExecutor: """Construct a Pandas agent from an LLM and dataframe(s). + Security Notice: + This agent relies on access to a python repl tool which can execute + arbitrary code. This can be dangerous and requires a specially sandboxed + environment to be safely used. Failure to run this code in a properly + sandboxed environment can lead to arbitrary code execution vulnerabilities, + which can lead to data breaches, data loss, or other security incidents. + + Do not use this code with untrusted inputs, with elevated permissions, + or without consulting your security team about proper sandboxing! + + You must opt-in to use this functionality by setting allow_dangerous_code=True. + Args: llm: Language model to use for the agent. If agent_type is "tool-calling" then llm is expected to support tool calling. @@ -198,6 +211,16 @@ def create_pandas_dataframe_agent( include_df_in_prompt is True. extra_tools: Additional tools to give to agent on top of a PythonAstREPLTool. engine: One of "modin" or "pandas". Defaults to "pandas". + allow_dangerous_code: bool, default False + This agent relies on access to a python repl tool which can execute + arbitrary code. This can be dangerous and requires a specially sandboxed + environment to be safely used. + Failure to properly sandbox this class can lead to arbitrary code execution + vulnerabilities, which can lead to data breaches, data loss, or + other security incidents. + You must opt in to use this functionality by setting + allow_dangerous_code=True. + **kwargs: DEPRECATED. Not used, kept for backwards compatibility. Returns: @@ -221,6 +244,16 @@ def create_pandas_dataframe_agent( ) """ + if not allow_dangerous_code: + raise ValueError( + "This agent relies on access to a python repl tool which can execute " + "arbitrary code. This can be dangerous and requires a specially sandboxed " + "environment to be safely used. Please read the security notice in the " + "doc-string of this function. You must opt-in to use this functionality " + "by setting allow_dangerous_code=True." + "For general security guidelines, please see: " + "https://python.langchain.com/v0.2/docs/security/" + ) try: if engine == "modin": import modin.pandas as pd diff --git a/libs/experimental/langchain_experimental/agents/agent_toolkits/spark/base.py b/libs/experimental/langchain_experimental/agents/agent_toolkits/spark/base.py index 8c09079c56765..e5a71188eea54 100644 --- a/libs/experimental/langchain_experimental/agents/agent_toolkits/spark/base.py +++ b/libs/experimental/langchain_experimental/agents/agent_toolkits/spark/base.py @@ -42,9 +42,44 @@ def create_spark_dataframe_agent( max_execution_time: Optional[float] = None, early_stopping_method: str = "force", agent_executor_kwargs: Optional[Dict[str, Any]] = None, + allow_dangerous_code: bool = False, **kwargs: Any, ) -> AgentExecutor: - """Construct a Spark agent from an LLM and dataframe.""" + """Construct a Spark agent from an LLM and dataframe. + + Security Notice: + This agent relies on access to a python repl tool which can execute + arbitrary code. This can be dangerous and requires a specially sandboxed + environment to be safely used. Failure to run this code in a properly + sandboxed environment can lead to arbitrary code execution vulnerabilities, + which can lead to data breaches, data loss, or other security incidents. + + Do not use this code with untrusted inputs, with elevated permissions, + or without consulting your security team about proper sandboxing! + + You must opt in to use this functionality by setting allow_dangerous_code=True. + + Args: + allow_dangerous_code: bool, default False + This agent relies on access to a python repl tool which can execute + arbitrary code. This can be dangerous and requires a specially sandboxed + environment to be safely used. + Failure to properly sandbox this class can lead to arbitrary code execution + vulnerabilities, which can lead to data breaches, data loss, or + other security incidents. + You must opt in to use this functionality by setting + allow_dangerous_code=True. + """ + if not allow_dangerous_code: + raise ValueError( + "This agent relies on access to a python repl tool which can execute " + "arbitrary code. This can be dangerous and requires a specially sandboxed " + "environment to be safely used. Please read the security notice in the " + "doc-string of this function. You must opt-in to use this functionality " + "by setting allow_dangerous_code=True." + "For general security guidelines, please see: " + "https://python.langchain.com/v0.2/docs/security/" + ) if not _validate_spark_df(df) and not _validate_spark_connect_df(df): raise ImportError("Spark is not installed. run `pip install pyspark`.") diff --git a/libs/experimental/langchain_experimental/agents/agent_toolkits/xorbits/base.py b/libs/experimental/langchain_experimental/agents/agent_toolkits/xorbits/base.py index 09f936401a6b8..d340f7b12ca6a 100644 --- a/libs/experimental/langchain_experimental/agents/agent_toolkits/xorbits/base.py +++ b/libs/experimental/langchain_experimental/agents/agent_toolkits/xorbits/base.py @@ -29,9 +29,45 @@ def create_xorbits_agent( max_execution_time: Optional[float] = None, early_stopping_method: str = "force", agent_executor_kwargs: Optional[Dict[str, Any]] = None, + allow_dangerous_code: bool = False, **kwargs: Dict[str, Any], ) -> AgentExecutor: - """Construct a xorbits agent from an LLM and dataframe.""" + """Construct a xorbits agent from an LLM and dataframe. + + Security Notice: + This agent relies on access to a python repl tool which can execute + arbitrary code. This can be dangerous and requires a specially sandboxed + environment to be safely used. Failure to run this code in a properly + sandboxed environment can lead to arbitrary code execution vulnerabilities, + which can lead to data breaches, data loss, or other security incidents. + + Do not use this code with untrusted inputs, with elevated permissions, + or without consulting your security team about proper sandboxing! + + You must opt in to use this functionality by setting allow_dangerous_code=True. + + Args: + allow_dangerous_code: bool, default False + This agent relies on access to a python repl tool which can execute + arbitrary code. This can be dangerous and requires a specially sandboxed + environment to be safely used. + Failure to properly sandbox this class can lead to arbitrary code execution + vulnerabilities, which can lead to data breaches, data loss, or + other security incidents. + You must opt in to use this functionality by setting + allow_dangerous_code=True. + """ + if not allow_dangerous_code: + raise ValueError( + "This agent relies on access to a python repl tool which can execute " + "arbitrary code. This can be dangerous and requires a specially sandboxed " + "environment to be safely used. Please read the security notice in the " + "doc-string of this function. You must opt-in to use this functionality " + "by setting allow_dangerous_code=True." + "For general security guidelines, please see: " + "https://python.langchain.com/v0.2/docs/security/" + ) + try: from xorbits import numpy as np from xorbits import pandas as pd diff --git a/libs/experimental/langchain_experimental/pal_chain/base.py b/libs/experimental/langchain_experimental/pal_chain/base.py index ad1fbc1c5de2d..3cb6a1621a436 100644 --- a/libs/experimental/langchain_experimental/pal_chain/base.py +++ b/libs/experimental/langchain_experimental/pal_chain/base.py @@ -18,7 +18,7 @@ from langchain_experimental.pal_chain.colored_object_prompt import COLORED_OBJECT_PROMPT from langchain_experimental.pal_chain.math_prompt import MATH_PROMPT -from langchain_experimental.pydantic_v1 import Extra, Field +from langchain_experimental.pydantic_v1 import Extra, Field, root_validator COMMAND_EXECUTION_FUNCTIONS = ["system", "exec", "execfile", "eval", "__import__"] COMMAND_EXECUTION_ATTRIBUTES = [ @@ -129,6 +129,36 @@ class PALChain(Chain): """Validations to perform on the generated code.""" timeout: Optional[int] = 10 """Timeout in seconds for the generated code to execute.""" + allow_dangerous_code: bool = False + """This chain relies on the execution of generated code, which can be dangerous. + + This class implements an AI technique that generates and evaluates + Python code, which can be dangerous and requires a specially sandboxed + environment to be safely used. While this class implements some basic guardrails + by limiting available locals/globals and by parsing and inspecting + the generated Python AST using `PALValidation`, those guardrails will not + deter sophisticated attackers and are not a replacement for a proper sandbox. + Do not use this class on untrusted inputs, with elevated permissions, + or without consulting your security team about proper sandboxing! + + Failure to properly sandbox this class can lead to arbitrary code execution + vulnerabilities, which can lead to data breaches, data loss, or other security + incidents. + """ + + @root_validator(pre=False, skip_on_failure=True) + def post_init(cls, values: Dict) -> Dict: + if not values["allow_dangerous_code"]: + raise ValueError( + "This chain relies on the execution of generated code, " + "which can be dangerous. " + "Please read the security notice for this class, and only " + "use it if you understand the security implications. " + "If you want to proceed, you will need to opt-in, by setting " + "`allow_dangerous_code` to `True`." + ) + + return values class Config: """Configuration for this pydantic object.""" diff --git a/libs/experimental/tests/integration_tests/chains/test_pal.py b/libs/experimental/tests/integration_tests/chains/test_pal.py index 1c623334a1ff8..ce58e3606c644 100644 --- a/libs/experimental/tests/integration_tests/chains/test_pal.py +++ b/libs/experimental/tests/integration_tests/chains/test_pal.py @@ -8,7 +8,7 @@ def test_math_prompt() -> None: """Test math prompt.""" llm = OpenAI(temperature=0, max_tokens=512) - pal_chain = PALChain.from_math_prompt(llm, timeout=None) + pal_chain = PALChain.from_math_prompt(llm, timeout=None, allow_dangerous_code=False) question = ( "Jan has three times the number of pets as Marcia. " "Marcia has two more pets than Cindy. " @@ -21,7 +21,9 @@ def test_math_prompt() -> None: def test_colored_object_prompt() -> None: """Test colored object prompt.""" llm = OpenAI(temperature=0, max_tokens=512) - pal_chain = PALChain.from_colored_object_prompt(llm, timeout=None) + pal_chain = PALChain.from_colored_object_prompt( + llm, timeout=None, allow_dangerous_code=False + ) question = ( "On the desk, you see two blue booklets, " "two purple booklets, and two yellow pairs of sunglasses. " diff --git a/libs/experimental/tests/unit_tests/agents/agent_toolkits/pandas/test_base.py b/libs/experimental/tests/unit_tests/agents/agent_toolkits/pandas/test_base.py index 8f3a063f6881e..9e99e4a7b5496 100644 --- a/libs/experimental/tests/unit_tests/agents/agent_toolkits/pandas/test_base.py +++ b/libs/experimental/tests/unit_tests/agents/agent_toolkits/pandas/test_base.py @@ -11,5 +11,12 @@ def test_create_pandas_dataframe_agent() -> None: import pandas as pd - create_pandas_dataframe_agent(FakeLLM(), pd.DataFrame()) - create_pandas_dataframe_agent(FakeLLM(), [pd.DataFrame(), pd.DataFrame()]) + with pytest.raises(ValueError): + create_pandas_dataframe_agent( + FakeLLM(), pd.DataFrame(), allow_dangerous_code=False + ) + + create_pandas_dataframe_agent(FakeLLM(), pd.DataFrame(), allow_dangerous_code=True) + create_pandas_dataframe_agent( + FakeLLM(), [pd.DataFrame(), pd.DataFrame()], allow_dangerous_code=True + ) diff --git a/libs/experimental/tests/unit_tests/test_pal.py b/libs/experimental/tests/unit_tests/test_pal.py index 0ad3f496e9dae..5c63cc5331c93 100644 --- a/libs/experimental/tests/unit_tests/test_pal.py +++ b/libs/experimental/tests/unit_tests/test_pal.py @@ -189,7 +189,9 @@ def test_math_question_1() -> None: prompt = MATH_PROMPT.format(question=question) queries = {prompt: _MATH_SOLUTION_1} fake_llm = FakeLLM(queries=queries) - fake_pal_chain = PALChain.from_math_prompt(fake_llm, timeout=None) + fake_pal_chain = PALChain.from_math_prompt( + fake_llm, timeout=None, allow_dangerous_code=True + ) output = fake_pal_chain.run(question) assert output == "8" @@ -202,7 +204,9 @@ def test_math_question_2() -> None: prompt = MATH_PROMPT.format(question=question) queries = {prompt: _MATH_SOLUTION_2} fake_llm = FakeLLM(queries=queries) - fake_pal_chain = PALChain.from_math_prompt(fake_llm, timeout=None) + fake_pal_chain = PALChain.from_math_prompt( + fake_llm, timeout=None, allow_dangerous_code=True + ) output = fake_pal_chain.run(question) assert output == "33" @@ -214,7 +218,9 @@ def test_math_question_3() -> None: prompt = MATH_PROMPT.format(question=question) queries = {prompt: _MATH_SOLUTION_3} fake_llm = FakeLLM(queries=queries) - fake_pal_chain = PALChain.from_math_prompt(fake_llm, timeout=None) + fake_pal_chain = PALChain.from_math_prompt( + fake_llm, timeout=None, allow_dangerous_code=True + ) with pytest.raises(ValueError) as exc_info: fake_pal_chain.run(question) assert ( @@ -231,7 +237,9 @@ def test_math_question_infinite_loop() -> None: prompt = MATH_PROMPT.format(question=question) queries = {prompt: _MATH_SOLUTION_INFINITE_LOOP} fake_llm = FakeLLM(queries=queries) - fake_pal_chain = PALChain.from_math_prompt(fake_llm, timeout=1) + fake_pal_chain = PALChain.from_math_prompt( + fake_llm, timeout=1, allow_dangerous_code=True + ) output = fake_pal_chain.run(question) assert output == "Execution timed out" @@ -245,7 +253,9 @@ def test_color_question_1() -> None: prompt = COLORED_OBJECT_PROMPT.format(question=question) queries = {prompt: _COLORED_OBJECT_SOLUTION_1} fake_llm = FakeLLM(queries=queries) - fake_pal_chain = PALChain.from_colored_object_prompt(fake_llm, timeout=None) + fake_pal_chain = PALChain.from_colored_object_prompt( + fake_llm, timeout=None, allow_dangerous_code=True + ) output = fake_pal_chain.run(question) assert output == "0" @@ -260,7 +270,9 @@ def test_color_question_2() -> None: prompt = COLORED_OBJECT_PROMPT.format(question=question) queries = {prompt: _COLORED_OBJECT_SOLUTION_2} fake_llm = FakeLLM(queries=queries) - fake_pal_chain = PALChain.from_colored_object_prompt(fake_llm, timeout=None) + fake_pal_chain = PALChain.from_colored_object_prompt( + fake_llm, timeout=None, allow_dangerous_code=True + ) output = fake_pal_chain.run(question) assert output == "brown" diff --git a/libs/partners/chroma/poetry.lock b/libs/partners/chroma/poetry.lock index ba2bc642819d4..d4e3af730c175 100644 --- a/libs/partners/chroma/poetry.lock +++ b/libs/partners/chroma/poetry.lock @@ -1,4 +1,4 @@ -# This file is automatically @generated by Poetry 1.7.1 and should not be changed by hand. +# This file is automatically @generated by Poetry 1.8.3 and should not be changed by hand. [[package]] name = "aiohttp" @@ -1288,7 +1288,7 @@ adal = ["adal (>=1.0.2)"] [[package]] name = "langchain" -version = "0.2.0rc2" +version = "0.2.3" description = "Building applications with LLMs through composability" optional = false python-versions = ">=3.8.1,<4.0" @@ -1298,39 +1298,26 @@ develop = true [package.dependencies] aiohttp = "^3.8.3" async-timeout = {version = "^4.0.0", markers = "python_version < \"3.11\""} -dataclasses-json = ">= 0.5.7, < 0.7" -langchain-core = ">=0.1.52,<0.3" -langchain-text-splitters = ">=0.0.1,<0.1" +langchain-core = "^0.2.0" +langchain-text-splitters = "^0.2.0" langsmith = "^0.1.17" -numpy = "^1" +numpy = [ + {version = ">=1,<2", markers = "python_version < \"3.12\""}, + {version = ">=1.26.0,<2.0.0", markers = "python_version >= \"3.12\""}, +] pydantic = ">=1,<3" PyYAML = ">=5.3" requests = "^2" SQLAlchemy = ">=1.4,<3" tenacity = "^8.1.0" -[package.extras] -all = [] -azure = ["azure-ai-formrecognizer (>=3.2.1,<4.0.0)", "azure-ai-textanalytics (>=5.3.0,<6.0.0)", "azure-cognitiveservices-speech (>=1.28.0,<2.0.0)", "azure-core (>=1.26.4,<2.0.0)", "azure-cosmos (>=4.4.0b1,<5.0.0)", "azure-identity (>=1.12.0,<2.0.0)", "azure-search-documents (==11.4.0b8)", "openai (<2)"] -clarifai = ["clarifai (>=9.1.0)"] -cli = ["typer (>=0.9.0,<0.10.0)"] -cohere = ["cohere (>=4,<6)"] -docarray = ["docarray[hnswlib] (>=0.32.0,<0.33.0)"] -embeddings = ["sentence-transformers (>=2,<3)"] -extended-testing = ["aiosqlite (>=0.19.0,<0.20.0)", "aleph-alpha-client (>=2.15.0,<3.0.0)", "anthropic (>=0.3.11,<0.4.0)", "arxiv (>=1.4,<2.0)", "assemblyai (>=0.17.0,<0.18.0)", "atlassian-python-api (>=3.36.0,<4.0.0)", "beautifulsoup4 (>=4,<5)", "bibtexparser (>=1.4.0,<2.0.0)", "cassio (>=0.1.0,<0.2.0)", "chardet (>=5.1.0,<6.0.0)", "cohere (>=4,<6)", "couchbase (>=4.1.9,<5.0.0)", "dashvector (>=1.0.1,<2.0.0)", "databricks-vectorsearch (>=0.21,<0.22)", "datasets (>=2.15.0,<3.0.0)", "dgml-utils (>=0.3.0,<0.4.0)", "esprima (>=4.0.1,<5.0.0)", "faiss-cpu (>=1,<2)", "feedparser (>=6.0.10,<7.0.0)", "fireworks-ai (>=0.9.0,<0.10.0)", "geopandas (>=0.13.1,<0.14.0)", "gitpython (>=3.1.32,<4.0.0)", "google-cloud-documentai (>=2.20.1,<3.0.0)", "gql (>=3.4.1,<4.0.0)", "hologres-vector (>=0.0.6,<0.0.7)", "html2text (>=2020.1.16,<2021.0.0)", "javelin-sdk (>=0.1.8,<0.2.0)", "jinja2 (>=3,<4)", "jq (>=1.4.1,<2.0.0)", "jsonschema (>1)", "langchain-openai (>=0.1,<0.2)", "lxml (>=4.9.3,<6.0)", "markdownify (>=0.11.6,<0.12.0)", "motor (>=3.3.1,<4.0.0)", "msal (>=1.25.0,<2.0.0)", "mwparserfromhell (>=0.6.4,<0.7.0)", "mwxml (>=0.3.3,<0.4.0)", "newspaper3k (>=0.2.8,<0.3.0)", "numexpr (>=2.8.6,<3.0.0)", "openai (<2)", "openai (<2)", "openapi-pydantic (>=0.3.2,<0.4.0)", "pandas (>=2.0.1,<3.0.0)", "pdfminer-six (>=20221105,<20221106)", "pgvector (>=0.1.6,<0.2.0)", "praw (>=7.7.1,<8.0.0)", "psychicapi (>=0.8.0,<0.9.0)", "py-trello (>=0.19.0,<0.20.0)", "pymupdf (>=1.22.3,<2.0.0)", "pypdf (>=3.4.0,<4.0.0)", "pypdfium2 (>=4.10.0,<5.0.0)", "pyspark (>=3.4.0,<4.0.0)", "rank-bm25 (>=0.2.2,<0.3.0)", "rapidfuzz (>=3.1.1,<4.0.0)", "rapidocr-onnxruntime (>=1.3.2,<2.0.0)", "rdflib (==7.0.0)", "requests-toolbelt (>=1.0.0,<2.0.0)", "rspace_client (>=2.5.0,<3.0.0)", "scikit-learn (>=1.2.2,<2.0.0)", "sqlite-vss (>=0.1.2,<0.2.0)", "streamlit (>=1.18.0,<2.0.0)", "sympy (>=1.12,<2.0)", "telethon (>=1.28.5,<2.0.0)", "timescale-vector (>=0.0.1,<0.0.2)", "tqdm (>=4.48.0)", "upstash-redis (>=0.15.0,<0.16.0)", "xata (>=1.0.0a7,<2.0.0)", "xmltodict (>=0.13.0,<0.14.0)"] -javascript = ["esprima (>=4.0.1,<5.0.0)"] -llms = ["clarifai (>=9.1.0)", "cohere (>=4,<6)", "huggingface_hub (>=0,<1)", "manifest-ml (>=0.0.1,<0.0.2)", "nlpcloud (>=1,<2)", "openai (<2)", "openlm (>=0.0.5,<0.0.6)", "torch (>=1,<3)", "transformers (>=4,<5)"] -openai = ["openai (<2)", "tiktoken (>=0.3.2,<0.6.0)"] -qdrant = ["qdrant-client (>=1.3.1,<2.0.0)"] -text-helpers = ["chardet (>=5.1.0,<6.0.0)"] - [package.source] type = "directory" url = "../../langchain" [[package]] name = "langchain-community" -version = "0.2.0rc1" +version = "0.2.4" description = "Community contributed LangChain integrations." optional = false python-versions = ">=3.8.1,<4.0" @@ -1340,26 +1327,25 @@ develop = true [package.dependencies] aiohttp = "^3.8.3" dataclasses-json = ">= 0.5.7, < 0.7" -langchain = "^0.2.0rc2" -langchain-core = ">=0.1.52,<0.3" +langchain = "^0.2.0" +langchain-core = "^0.2.0" langsmith = "^0.1.0" -numpy = "^1" +numpy = [ + {version = ">=1,<2", markers = "python_version < \"3.12\""}, + {version = ">=1.26.0,<2.0.0", markers = "python_version >= \"3.12\""}, +] PyYAML = ">=5.3" requests = "^2" SQLAlchemy = ">=1.4,<3" tenacity = "^8.1.0" -[package.extras] -cli = ["typer (>=0.9.0,<0.10.0)"] -extended-testing = ["aiosqlite (>=0.19.0,<0.20.0)", "aleph-alpha-client (>=2.15.0,<3.0.0)", "anthropic (>=0.3.11,<0.4.0)", "arxiv (>=1.4,<2.0)", "assemblyai (>=0.17.0,<0.18.0)", "atlassian-python-api (>=3.36.0,<4.0.0)", "azure-ai-documentintelligence (>=1.0.0b1,<2.0.0)", "azure-identity (>=1.15.0,<2.0.0)", "azure-search-documents (==11.4.0)", "beautifulsoup4 (>=4,<5)", "bibtexparser (>=1.4.0,<2.0.0)", "cassio (>=0.1.6,<0.2.0)", "chardet (>=5.1.0,<6.0.0)", "cloudpickle (>=2.0.0)", "cloudpickle (>=2.0.0)", "cohere (>=4,<5)", "databricks-vectorsearch (>=0.21,<0.22)", "datasets (>=2.15.0,<3.0.0)", "dgml-utils (>=0.3.0,<0.4.0)", "elasticsearch (>=8.12.0,<9.0.0)", "esprima (>=4.0.1,<5.0.0)", "faiss-cpu (>=1,<2)", "feedparser (>=6.0.10,<7.0.0)", "fireworks-ai (>=0.9.0,<0.10.0)", "friendli-client (>=1.2.4,<2.0.0)", "geopandas (>=0.13.1,<0.14.0)", "gitpython (>=3.1.32,<4.0.0)", "google-cloud-documentai (>=2.20.1,<3.0.0)", "gql (>=3.4.1,<4.0.0)", "gradientai (>=1.4.0,<2.0.0)", "hdbcli (>=2.19.21,<3.0.0)", "hologres-vector (>=0.0.6,<0.0.7)", "html2text (>=2020.1.16,<2021.0.0)", "httpx (>=0.24.1,<0.25.0)", "httpx-sse (>=0.4.0,<0.5.0)", "javelin-sdk (>=0.1.8,<0.2.0)", "jinja2 (>=3,<4)", "jq (>=1.4.1,<2.0.0)", "jsonschema (>1)", "lxml (>=4.9.3,<6.0)", "markdownify (>=0.11.6,<0.12.0)", "motor (>=3.3.1,<4.0.0)", "msal (>=1.25.0,<2.0.0)", "mwparserfromhell (>=0.6.4,<0.7.0)", "mwxml (>=0.3.3,<0.4.0)", "newspaper3k (>=0.2.8,<0.3.0)", "numexpr (>=2.8.6,<3.0.0)", "nvidia-riva-client (>=2.14.0,<3.0.0)", "oci (>=2.119.1,<3.0.0)", "openai (<2)", "openapi-pydantic (>=0.3.2,<0.4.0)", "oracle-ads (>=2.9.1,<3.0.0)", "oracledb (>=2.2.0,<3.0.0)", "pandas (>=2.0.1,<3.0.0)", "pdfminer-six (>=20221105,<20221106)", "pgvector (>=0.1.6,<0.2.0)", "praw (>=7.7.1,<8.0.0)", "premai (>=0.3.25,<0.4.0)", "psychicapi (>=0.8.0,<0.9.0)", "py-trello (>=0.19.0,<0.20.0)", "pyjwt (>=2.8.0,<3.0.0)", "pymupdf (>=1.22.3,<2.0.0)", "pypdf (>=3.4.0,<4.0.0)", "pypdfium2 (>=4.10.0,<5.0.0)", "pyspark (>=3.4.0,<4.0.0)", "rank-bm25 (>=0.2.2,<0.3.0)", "rapidfuzz (>=3.1.1,<4.0.0)", "rapidocr-onnxruntime (>=1.3.2,<2.0.0)", "rdflib (==7.0.0)", "requests-toolbelt (>=1.0.0,<2.0.0)", "rspace_client (>=2.5.0,<3.0.0)", "scikit-learn (>=1.2.2,<2.0.0)", "sqlite-vss (>=0.1.2,<0.2.0)", "streamlit (>=1.18.0,<2.0.0)", "sympy (>=1.12,<2.0)", "telethon (>=1.28.5,<2.0.0)", "tidb-vector (>=0.0.3,<1.0.0)", "timescale-vector (>=0.0.1,<0.0.2)", "tqdm (>=4.48.0)", "tree-sitter (>=0.20.2,<0.21.0)", "tree-sitter-languages (>=1.8.0,<2.0.0)", "upstash-redis (>=0.15.0,<0.16.0)", "vdms (>=0.0.20,<0.0.21)", "xata (>=1.0.0a7,<2.0.0)", "xmltodict (>=0.13.0,<0.14.0)"] - [package.source] type = "directory" url = "../../community" [[package]] name = "langchain-core" -version = "0.2.0rc1" +version = "0.2.5" description = "Building applications with LLMs through composability" optional = false python-versions = ">=3.8.1,<4.0" @@ -1368,22 +1354,19 @@ develop = true [package.dependencies] jsonpatch = "^1.33" -langsmith = "^0.1.0" -packaging = "^23.2" +langsmith = "^0.1.75" +packaging = ">=23.2,<25" pydantic = ">=1,<3" PyYAML = ">=5.3" tenacity = "^8.1.0" -[package.extras] -extended-testing = ["jinja2 (>=3,<4)"] - [package.source] type = "directory" url = "../../core" [[package]] name = "langchain-openai" -version = "0.1.7" +version = "0.1.8" description = "An integration package connecting OpenAI and LangChain" optional = false python-versions = ">=3.8.1,<4.0" @@ -1391,9 +1374,9 @@ files = [] develop = true [package.dependencies] -langchain-core = ">=0.1.46,<0.3" -openai = "^1.24.0" -tiktoken = ">=0.5.2,<1" +langchain-core = ">=0.2.2,<0.3" +openai = "^1.26.0" +tiktoken = ">=0.7,<1" [package.source] type = "directory" @@ -1401,7 +1384,7 @@ url = "../openai" [[package]] name = "langchain-text-splitters" -version = "0.0.2" +version = "0.2.1" description = "LangChain text splitting utilities" optional = false python-versions = ">=3.8.1,<4.0" @@ -1409,10 +1392,7 @@ files = [] develop = true [package.dependencies] -langchain-core = ">=0.1.28,<0.3" - -[package.extras] -extended-testing = ["beautifulsoup4 (>=4.12.3,<5.0.0)", "lxml (>=4.9.3,<6.0)"] +langchain-core = "^0.2.0" [package.source] type = "directory" @@ -1420,13 +1400,13 @@ url = "../../text-splitters" [[package]] name = "langsmith" -version = "0.1.58" +version = "0.1.77" description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform." optional = false python-versions = "<4.0,>=3.8.1" files = [ - {file = "langsmith-0.1.58-py3-none-any.whl", hash = "sha256:1148cc836ec99d1b2f37cd2fa3014fcac213bb6bad798a2b21bb9111c18c9768"}, - {file = "langsmith-0.1.58.tar.gz", hash = "sha256:a5060933c1fb3006b498ec849677993329d7e6138bdc2ec044068ab806e09c39"}, + {file = "langsmith-0.1.77-py3-none-any.whl", hash = "sha256:2202cc21b1ed7e7b9e5d2af2694be28898afa048c09fdf09f620cbd9301755ae"}, + {file = "langsmith-0.1.77.tar.gz", hash = "sha256:4ace09077a9a4e412afeb4b517ca68e7de7b07f36e4792dc8236ac5207c0c0c7"}, ] [package.dependencies] @@ -1871,6 +1851,51 @@ files = [ {file = "numpy-1.24.4.tar.gz", hash = "sha256:80f5e3a4e498641401868df4208b74581206afbee7cf7b8329daae82676d9463"}, ] +[[package]] +name = "numpy" +version = "1.26.4" +description = "Fundamental package for array computing in Python" +optional = false +python-versions = ">=3.9" +files = [ + {file = "numpy-1.26.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9ff0f4f29c51e2803569d7a51c2304de5554655a60c5d776e35b4a41413830d0"}, + {file = "numpy-1.26.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:2e4ee3380d6de9c9ec04745830fd9e2eccb3e6cf790d39d7b98ffd19b0dd754a"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d209d8969599b27ad20994c8e41936ee0964e6da07478d6c35016bc386b66ad4"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ffa75af20b44f8dba823498024771d5ac50620e6915abac414251bd971b4529f"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:62b8e4b1e28009ef2846b4c7852046736bab361f7aeadeb6a5b89ebec3c7055a"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:a4abb4f9001ad2858e7ac189089c42178fcce737e4169dc61321660f1a96c7d2"}, + {file = "numpy-1.26.4-cp310-cp310-win32.whl", hash = "sha256:bfe25acf8b437eb2a8b2d49d443800a5f18508cd811fea3181723922a8a82b07"}, + {file = "numpy-1.26.4-cp310-cp310-win_amd64.whl", hash = "sha256:b97fe8060236edf3662adfc2c633f56a08ae30560c56310562cb4f95500022d5"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:4c66707fabe114439db9068ee468c26bbdf909cac0fb58686a42a24de1760c71"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:edd8b5fe47dab091176d21bb6de568acdd906d1887a4584a15a9a96a1dca06ef"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7ab55401287bfec946ced39700c053796e7cc0e3acbef09993a9ad2adba6ca6e"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:666dbfb6ec68962c033a450943ded891bed2d54e6755e35e5835d63f4f6931d5"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:96ff0b2ad353d8f990b63294c8986f1ec3cb19d749234014f4e7eb0112ceba5a"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:60dedbb91afcbfdc9bc0b1f3f402804070deed7392c23eb7a7f07fa857868e8a"}, + {file = "numpy-1.26.4-cp311-cp311-win32.whl", hash = "sha256:1af303d6b2210eb850fcf03064d364652b7120803a0b872f5211f5234b399f20"}, + {file = "numpy-1.26.4-cp311-cp311-win_amd64.whl", hash = "sha256:cd25bcecc4974d09257ffcd1f098ee778f7834c3ad767fe5db785be9a4aa9cb2"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:b3ce300f3644fb06443ee2222c2201dd3a89ea6040541412b8fa189341847218"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:03a8c78d01d9781b28a6989f6fa1bb2c4f2d51201cf99d3dd875df6fbd96b23b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9fad7dcb1aac3c7f0584a5a8133e3a43eeb2fe127f47e3632d43d677c66c102b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:675d61ffbfa78604709862923189bad94014bef562cc35cf61d3a07bba02a7ed"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:ab47dbe5cc8210f55aa58e4805fe224dac469cde56b9f731a4c098b91917159a"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:1dda2e7b4ec9dd512f84935c5f126c8bd8b9f2fc001e9f54af255e8c5f16b0e0"}, + {file = "numpy-1.26.4-cp312-cp312-win32.whl", hash = "sha256:50193e430acfc1346175fcbdaa28ffec49947a06918b7b92130744e81e640110"}, + {file = "numpy-1.26.4-cp312-cp312-win_amd64.whl", hash = "sha256:08beddf13648eb95f8d867350f6a018a4be2e5ad54c8d8caed89ebca558b2818"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:7349ab0fa0c429c82442a27a9673fc802ffdb7c7775fad780226cb234965e53c"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:52b8b60467cd7dd1e9ed082188b4e6bb35aa5cdd01777621a1658910745b90be"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d5241e0a80d808d70546c697135da2c613f30e28251ff8307eb72ba696945764"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f870204a840a60da0b12273ef34f7051e98c3b5961b61b0c2c1be6dfd64fbcd3"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:679b0076f67ecc0138fd2ede3a8fd196dddc2ad3254069bcb9faf9a79b1cebcd"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:47711010ad8555514b434df65f7d7b076bb8261df1ca9bb78f53d3b2db02e95c"}, + {file = "numpy-1.26.4-cp39-cp39-win32.whl", hash = "sha256:a354325ee03388678242a4d7ebcd08b5c727033fcff3b2f536aea978e15ee9e6"}, + {file = "numpy-1.26.4-cp39-cp39-win_amd64.whl", hash = "sha256:3373d5d70a5fe74a2c1bb6d2cfd9609ecf686d47a2d7b1d37a8f3b6bf6003aea"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:afedb719a9dcfc7eaf2287b839d8198e06dcd4cb5d276a3df279231138e83d30"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:95a7476c59002f2f6c590b9b7b998306fba6a5aa646b1e22ddfeaf8f78c3a29c"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:7e50d0a0cc3189f9cb0aeb3a6a6af18c16f59f004b866cd2be1c14b36134a4a0"}, + {file = "numpy-1.26.4.tar.gz", hash = "sha256:2a02aba9ed12e4ac4eb3ea9421c420301a0c6460d9830d74a9df87efa4912010"}, +] + [[package]] name = "oauthlib" version = "3.2.2" @@ -3843,4 +3868,4 @@ testing = ["big-O", "jaraco.functools", "jaraco.itertools", "more-itertools", "p [metadata] lock-version = "2.0" python-versions = ">=3.8.1,<3.13" -content-hash = "0d330760df4899b6f276fa66c58466e088d27df1c12f51228d177c2155a8ec9d" +content-hash = "ae5ce141d47598fc2656a1be163a5c9b889ebb7e8a77b2a739d4c9c2b9c3dc4c" diff --git a/libs/partners/chroma/pyproject.toml b/libs/partners/chroma/pyproject.toml index e4501d1b0f9c9..4723da18c356c 100644 --- a/libs/partners/chroma/pyproject.toml +++ b/libs/partners/chroma/pyproject.toml @@ -14,7 +14,11 @@ license = "MIT" python = ">=3.8.1,<3.13" langchain-core = ">=0.1.40,<0.3" chromadb = { version = ">=0.4.0,<0.6.0" } -numpy = "^1" +# Support Python 3.8 and 3.12+. +numpy = [ + {version = "^1", python = "<3.12"}, + {version = "^1.26.0", python = ">=3.12"} +] fastapi = { version = ">=0.95.2,<1", optional = true } [tool.poetry.group.test] diff --git a/libs/partners/mongodb/poetry.lock b/libs/partners/mongodb/poetry.lock index 93a8d2fa4f414..6dd7637ccd0a7 100644 --- a/libs/partners/mongodb/poetry.lock +++ b/libs/partners/mongodb/poetry.lock @@ -1,4 +1,4 @@ -# This file is automatically @generated by Poetry 1.7.1 and should not be changed by hand. +# This file is automatically @generated by Poetry 1.8.3 and should not be changed by hand. [[package]] name = "aiohttp" @@ -292,21 +292,6 @@ files = [ {file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"}, ] -[[package]] -name = "dataclasses-json" -version = "0.6.6" -description = "Easily serialize dataclasses to and from JSON." -optional = false -python-versions = "<4.0,>=3.7" -files = [ - {file = "dataclasses_json-0.6.6-py3-none-any.whl", hash = "sha256:e54c5c87497741ad454070ba0ed411523d46beb5da102e221efb873801b0ba85"}, - {file = "dataclasses_json-0.6.6.tar.gz", hash = "sha256:0c09827d26fffda27f1be2fed7a7a01a29c5ddcd2eb6393ad5ebf9d77e9deae8"}, -] - -[package.dependencies] -marshmallow = ">=3.18.0,<4.0.0" -typing-inspect = ">=0.4.0,<1" - [[package]] name = "dnspython" version = "2.6.1" @@ -561,7 +546,7 @@ files = [ [[package]] name = "langchain" -version = "0.2.0rc2" +version = "0.2.3" description = "Building applications with LLMs through composability" optional = false python-versions = ">=3.8.1,<4.0" @@ -571,39 +556,26 @@ develop = true [package.dependencies] aiohttp = "^3.8.3" async-timeout = {version = "^4.0.0", markers = "python_version < \"3.11\""} -dataclasses-json = ">= 0.5.7, < 0.7" -langchain-core = ">=0.1.52,<0.3" -langchain-text-splitters = ">=0.0.1,<0.1" +langchain-core = "^0.2.0" +langchain-text-splitters = "^0.2.0" langsmith = "^0.1.17" -numpy = "^1" +numpy = [ + {version = ">=1,<2", markers = "python_version < \"3.12\""}, + {version = ">=1.26.0,<2.0.0", markers = "python_version >= \"3.12\""}, +] pydantic = ">=1,<3" PyYAML = ">=5.3" requests = "^2" SQLAlchemy = ">=1.4,<3" tenacity = "^8.1.0" -[package.extras] -all = [] -azure = ["azure-ai-formrecognizer (>=3.2.1,<4.0.0)", "azure-ai-textanalytics (>=5.3.0,<6.0.0)", "azure-cognitiveservices-speech (>=1.28.0,<2.0.0)", "azure-core (>=1.26.4,<2.0.0)", "azure-cosmos (>=4.4.0b1,<5.0.0)", "azure-identity (>=1.12.0,<2.0.0)", "azure-search-documents (==11.4.0b8)", "openai (<2)"] -clarifai = ["clarifai (>=9.1.0)"] -cli = ["typer (>=0.9.0,<0.10.0)"] -cohere = ["cohere (>=4,<6)"] -docarray = ["docarray[hnswlib] (>=0.32.0,<0.33.0)"] -embeddings = ["sentence-transformers (>=2,<3)"] -extended-testing = ["aiosqlite (>=0.19.0,<0.20.0)", "aleph-alpha-client (>=2.15.0,<3.0.0)", "anthropic (>=0.3.11,<0.4.0)", "arxiv (>=1.4,<2.0)", "assemblyai (>=0.17.0,<0.18.0)", "atlassian-python-api (>=3.36.0,<4.0.0)", "beautifulsoup4 (>=4,<5)", "bibtexparser (>=1.4.0,<2.0.0)", "cassio (>=0.1.0,<0.2.0)", "chardet (>=5.1.0,<6.0.0)", "cohere (>=4,<6)", "couchbase (>=4.1.9,<5.0.0)", "dashvector (>=1.0.1,<2.0.0)", "databricks-vectorsearch (>=0.21,<0.22)", "datasets (>=2.15.0,<3.0.0)", "dgml-utils (>=0.3.0,<0.4.0)", "esprima (>=4.0.1,<5.0.0)", "faiss-cpu (>=1,<2)", "feedparser (>=6.0.10,<7.0.0)", "fireworks-ai (>=0.9.0,<0.10.0)", "geopandas (>=0.13.1,<0.14.0)", "gitpython (>=3.1.32,<4.0.0)", "google-cloud-documentai (>=2.20.1,<3.0.0)", "gql (>=3.4.1,<4.0.0)", "hologres-vector (>=0.0.6,<0.0.7)", "html2text (>=2020.1.16,<2021.0.0)", "javelin-sdk (>=0.1.8,<0.2.0)", "jinja2 (>=3,<4)", "jq (>=1.4.1,<2.0.0)", "jsonschema (>1)", "langchain-openai (>=0.1,<0.2)", "lxml (>=4.9.3,<6.0)", "markdownify (>=0.11.6,<0.12.0)", "motor (>=3.3.1,<4.0.0)", "msal (>=1.25.0,<2.0.0)", "mwparserfromhell (>=0.6.4,<0.7.0)", "mwxml (>=0.3.3,<0.4.0)", "newspaper3k (>=0.2.8,<0.3.0)", "numexpr (>=2.8.6,<3.0.0)", "openai (<2)", "openai (<2)", "openapi-pydantic (>=0.3.2,<0.4.0)", "pandas (>=2.0.1,<3.0.0)", "pdfminer-six (>=20221105,<20221106)", "pgvector (>=0.1.6,<0.2.0)", "praw (>=7.7.1,<8.0.0)", "psychicapi (>=0.8.0,<0.9.0)", "py-trello (>=0.19.0,<0.20.0)", "pymupdf (>=1.22.3,<2.0.0)", "pypdf (>=3.4.0,<4.0.0)", "pypdfium2 (>=4.10.0,<5.0.0)", "pyspark (>=3.4.0,<4.0.0)", "rank-bm25 (>=0.2.2,<0.3.0)", "rapidfuzz (>=3.1.1,<4.0.0)", "rapidocr-onnxruntime (>=1.3.2,<2.0.0)", "rdflib (==7.0.0)", "requests-toolbelt (>=1.0.0,<2.0.0)", "rspace_client (>=2.5.0,<3.0.0)", "scikit-learn (>=1.2.2,<2.0.0)", "sqlite-vss (>=0.1.2,<0.2.0)", "streamlit (>=1.18.0,<2.0.0)", "sympy (>=1.12,<2.0)", "telethon (>=1.28.5,<2.0.0)", "timescale-vector (>=0.0.1,<0.0.2)", "tqdm (>=4.48.0)", "upstash-redis (>=0.15.0,<0.16.0)", "xata (>=1.0.0a7,<2.0.0)", "xmltodict (>=0.13.0,<0.14.0)"] -javascript = ["esprima (>=4.0.1,<5.0.0)"] -llms = ["clarifai (>=9.1.0)", "cohere (>=4,<6)", "huggingface_hub (>=0,<1)", "manifest-ml (>=0.0.1,<0.0.2)", "nlpcloud (>=1,<2)", "openai (<2)", "openlm (>=0.0.5,<0.0.6)", "torch (>=1,<3)", "transformers (>=4,<5)"] -openai = ["openai (<2)", "tiktoken (>=0.3.2,<0.6.0)"] -qdrant = ["qdrant-client (>=1.3.1,<2.0.0)"] -text-helpers = ["chardet (>=5.1.0,<6.0.0)"] - [package.source] type = "directory" url = "../../langchain" [[package]] name = "langchain-core" -version = "0.2.0rc1" +version = "0.2.5" description = "Building applications with LLMs through composability" optional = false python-versions = ">=3.8.1,<4.0" @@ -612,22 +584,19 @@ develop = true [package.dependencies] jsonpatch = "^1.33" -langsmith = "^0.1.0" -packaging = "^23.2" +langsmith = "^0.1.75" +packaging = ">=23.2,<25" pydantic = ">=1,<3" PyYAML = ">=5.3" tenacity = "^8.1.0" -[package.extras] -extended-testing = ["jinja2 (>=3,<4)"] - [package.source] type = "directory" url = "../../core" [[package]] name = "langchain-text-splitters" -version = "0.0.2" +version = "0.2.1" description = "LangChain text splitting utilities" optional = false python-versions = ">=3.8.1,<4.0" @@ -635,10 +604,7 @@ files = [] develop = true [package.dependencies] -langchain-core = ">=0.1.28,<0.3" - -[package.extras] -extended-testing = ["beautifulsoup4 (>=4.12.3,<5.0.0)", "lxml (>=4.9.3,<6.0)"] +langchain-core = "^0.2.0" [package.source] type = "directory" @@ -646,13 +612,13 @@ url = "../../text-splitters" [[package]] name = "langsmith" -version = "0.1.58" +version = "0.1.77" description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform." optional = false python-versions = "<4.0,>=3.8.1" files = [ - {file = "langsmith-0.1.58-py3-none-any.whl", hash = "sha256:1148cc836ec99d1b2f37cd2fa3014fcac213bb6bad798a2b21bb9111c18c9768"}, - {file = "langsmith-0.1.58.tar.gz", hash = "sha256:a5060933c1fb3006b498ec849677993329d7e6138bdc2ec044068ab806e09c39"}, + {file = "langsmith-0.1.77-py3-none-any.whl", hash = "sha256:2202cc21b1ed7e7b9e5d2af2694be28898afa048c09fdf09f620cbd9301755ae"}, + {file = "langsmith-0.1.77.tar.gz", hash = "sha256:4ace09077a9a4e412afeb4b517ca68e7de7b07f36e4792dc8236ac5207c0c0c7"}, ] [package.dependencies] @@ -660,25 +626,6 @@ orjson = ">=3.9.14,<4.0.0" pydantic = ">=1,<3" requests = ">=2,<3" -[[package]] -name = "marshmallow" -version = "3.21.2" -description = "A lightweight library for converting complex datatypes to and from native Python datatypes." -optional = false -python-versions = ">=3.8" -files = [ - {file = "marshmallow-3.21.2-py3-none-any.whl", hash = "sha256:70b54a6282f4704d12c0a41599682c5c5450e843b9ec406308653b47c59648a1"}, - {file = "marshmallow-3.21.2.tar.gz", hash = "sha256:82408deadd8b33d56338d2182d455db632c6313aa2af61916672146bb32edc56"}, -] - -[package.dependencies] -packaging = ">=17.0" - -[package.extras] -dev = ["marshmallow[tests]", "pre-commit (>=3.5,<4.0)", "tox"] -docs = ["alabaster (==0.7.16)", "autodocsumm (==0.2.12)", "sphinx (==7.3.7)", "sphinx-issues (==4.1.0)", "sphinx-version-warning (==1.1.2)"] -tests = ["pytest", "pytz", "simplejson"] - [[package]] name = "multidict" version = "6.0.5" @@ -876,6 +823,51 @@ files = [ {file = "numpy-1.24.4.tar.gz", hash = "sha256:80f5e3a4e498641401868df4208b74581206afbee7cf7b8329daae82676d9463"}, ] +[[package]] +name = "numpy" +version = "1.26.4" +description = "Fundamental package for array computing in Python" +optional = false +python-versions = ">=3.9" +files = [ + {file = "numpy-1.26.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9ff0f4f29c51e2803569d7a51c2304de5554655a60c5d776e35b4a41413830d0"}, + {file = "numpy-1.26.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:2e4ee3380d6de9c9ec04745830fd9e2eccb3e6cf790d39d7b98ffd19b0dd754a"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d209d8969599b27ad20994c8e41936ee0964e6da07478d6c35016bc386b66ad4"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ffa75af20b44f8dba823498024771d5ac50620e6915abac414251bd971b4529f"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:62b8e4b1e28009ef2846b4c7852046736bab361f7aeadeb6a5b89ebec3c7055a"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:a4abb4f9001ad2858e7ac189089c42178fcce737e4169dc61321660f1a96c7d2"}, + {file = "numpy-1.26.4-cp310-cp310-win32.whl", hash = "sha256:bfe25acf8b437eb2a8b2d49d443800a5f18508cd811fea3181723922a8a82b07"}, + {file = "numpy-1.26.4-cp310-cp310-win_amd64.whl", hash = "sha256:b97fe8060236edf3662adfc2c633f56a08ae30560c56310562cb4f95500022d5"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:4c66707fabe114439db9068ee468c26bbdf909cac0fb58686a42a24de1760c71"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:edd8b5fe47dab091176d21bb6de568acdd906d1887a4584a15a9a96a1dca06ef"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7ab55401287bfec946ced39700c053796e7cc0e3acbef09993a9ad2adba6ca6e"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:666dbfb6ec68962c033a450943ded891bed2d54e6755e35e5835d63f4f6931d5"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:96ff0b2ad353d8f990b63294c8986f1ec3cb19d749234014f4e7eb0112ceba5a"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:60dedbb91afcbfdc9bc0b1f3f402804070deed7392c23eb7a7f07fa857868e8a"}, + {file = "numpy-1.26.4-cp311-cp311-win32.whl", hash = "sha256:1af303d6b2210eb850fcf03064d364652b7120803a0b872f5211f5234b399f20"}, + {file = "numpy-1.26.4-cp311-cp311-win_amd64.whl", hash = "sha256:cd25bcecc4974d09257ffcd1f098ee778f7834c3ad767fe5db785be9a4aa9cb2"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:b3ce300f3644fb06443ee2222c2201dd3a89ea6040541412b8fa189341847218"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:03a8c78d01d9781b28a6989f6fa1bb2c4f2d51201cf99d3dd875df6fbd96b23b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9fad7dcb1aac3c7f0584a5a8133e3a43eeb2fe127f47e3632d43d677c66c102b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:675d61ffbfa78604709862923189bad94014bef562cc35cf61d3a07bba02a7ed"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:ab47dbe5cc8210f55aa58e4805fe224dac469cde56b9f731a4c098b91917159a"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:1dda2e7b4ec9dd512f84935c5f126c8bd8b9f2fc001e9f54af255e8c5f16b0e0"}, + {file = "numpy-1.26.4-cp312-cp312-win32.whl", hash = "sha256:50193e430acfc1346175fcbdaa28ffec49947a06918b7b92130744e81e640110"}, + {file = "numpy-1.26.4-cp312-cp312-win_amd64.whl", hash = "sha256:08beddf13648eb95f8d867350f6a018a4be2e5ad54c8d8caed89ebca558b2818"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:7349ab0fa0c429c82442a27a9673fc802ffdb7c7775fad780226cb234965e53c"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:52b8b60467cd7dd1e9ed082188b4e6bb35aa5cdd01777621a1658910745b90be"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d5241e0a80d808d70546c697135da2c613f30e28251ff8307eb72ba696945764"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f870204a840a60da0b12273ef34f7051e98c3b5961b61b0c2c1be6dfd64fbcd3"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:679b0076f67ecc0138fd2ede3a8fd196dddc2ad3254069bcb9faf9a79b1cebcd"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:47711010ad8555514b434df65f7d7b076bb8261df1ca9bb78f53d3b2db02e95c"}, + {file = "numpy-1.26.4-cp39-cp39-win32.whl", hash = "sha256:a354325ee03388678242a4d7ebcd08b5c727033fcff3b2f536aea978e15ee9e6"}, + {file = "numpy-1.26.4-cp39-cp39-win_amd64.whl", hash = "sha256:3373d5d70a5fe74a2c1bb6d2cfd9609ecf686d47a2d7b1d37a8f3b6bf6003aea"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:afedb719a9dcfc7eaf2287b839d8198e06dcd4cb5d276a3df279231138e83d30"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:95a7476c59002f2f6c590b9b7b998306fba6a5aa646b1e22ddfeaf8f78c3a29c"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:7e50d0a0cc3189f9cb0aeb3a6a6af18c16f59f004b866cd2be1c14b36134a4a0"}, + {file = "numpy-1.26.4.tar.gz", hash = "sha256:2a02aba9ed12e4ac4eb3ea9421c420301a0c6460d9830d74a9df87efa4912010"}, +] + [[package]] name = "orjson" version = "3.10.3" @@ -1490,21 +1482,6 @@ files = [ {file = "typing_extensions-4.11.0.tar.gz", hash = "sha256:83f085bd5ca59c80295fc2a82ab5dac679cbe02b9f33f7d83af68e241bea51b0"}, ] -[[package]] -name = "typing-inspect" -version = "0.9.0" -description = "Runtime inspection utilities for typing module." -optional = false -python-versions = "*" -files = [ - {file = "typing_inspect-0.9.0-py3-none-any.whl", hash = "sha256:9ee6fc59062311ef8547596ab6b955e1b8aa46242d854bfc78f4f6b0eff35f9f"}, - {file = "typing_inspect-0.9.0.tar.gz", hash = "sha256:b23fc42ff6f6ef6954e4852c1fb512cdd18dbea03134f91f856a95ccc9461f78"}, -] - -[package.dependencies] -mypy-extensions = ">=0.3.0" -typing-extensions = ">=3.7.4" - [[package]] name = "urllib3" version = "2.2.1" @@ -1669,4 +1646,4 @@ multidict = ">=4.0" [metadata] lock-version = "2.0" python-versions = ">=3.8.1,<4.0" -content-hash = "40a8f66340dfd1b2de64acdcc8daf1359040f90a8eb601992c533a7380dd3da3" +content-hash = "48cee695a3fc0745a7863094f7463ed3526ba04fbe40be17d242e22a36d756f1" diff --git a/libs/partners/mongodb/pyproject.toml b/libs/partners/mongodb/pyproject.toml index 26de58d289678..b313af7b88257 100644 --- a/libs/partners/mongodb/pyproject.toml +++ b/libs/partners/mongodb/pyproject.toml @@ -14,7 +14,11 @@ license = "MIT" python = ">=3.8.1,<4.0" pymongo = ">=4.6.1,<5.0" langchain-core = ">=0.1.46,<0.3" -numpy = "^1" +# Support Python 3.8 and 3.12+. +numpy = [ + {version = "^1", python = "<3.12"}, + {version = "^1.26.0", python = ">=3.12"} +] [tool.poetry.group.test] optional = true diff --git a/libs/partners/openai/poetry.lock b/libs/partners/openai/poetry.lock index b7fdaffce8121..1f62bdc1062bc 100644 --- a/libs/partners/openai/poetry.lock +++ b/libs/partners/openai/poetry.lock @@ -385,7 +385,7 @@ files = [ [[package]] name = "langchain-core" -version = "0.2.2" +version = "0.2.5" description = "Building applications with LLMs through composability" optional = false python-versions = ">=3.8.1,<4.0" @@ -394,15 +394,12 @@ develop = true [package.dependencies] jsonpatch = "^1.33" -langsmith = "^0.1.0" -packaging = "^23.2" +langsmith = "^0.1.75" +packaging = ">=23.2,<25" pydantic = ">=1,<3" PyYAML = ">=5.3" tenacity = "^8.1.0" -[package.extras] -extended-testing = ["jinja2 (>=3,<4)"] - [package.source] type = "directory" url = "../../core" @@ -426,13 +423,13 @@ url = "../../standard-tests" [[package]] name = "langsmith" -version = "0.1.57" +version = "0.1.77" description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform." optional = false python-versions = "<4.0,>=3.8.1" files = [ - {file = "langsmith-0.1.57-py3-none-any.whl", hash = "sha256:dbd83b0944a2fbea4151f0aa053530d93fcf6784a580621bc60633cb890b57dc"}, - {file = "langsmith-0.1.57.tar.gz", hash = "sha256:4682204de19f0218029c2b8445ce2cc3485c8d0df9796b31e2ce4c9051fce365"}, + {file = "langsmith-0.1.77-py3-none-any.whl", hash = "sha256:2202cc21b1ed7e7b9e5d2af2694be28898afa048c09fdf09f620cbd9301755ae"}, + {file = "langsmith-0.1.77.tar.gz", hash = "sha256:4ace09077a9a4e412afeb4b517ca68e7de7b07f36e4792dc8236ac5207c0c0c7"}, ] [package.dependencies] @@ -538,6 +535,51 @@ files = [ {file = "numpy-1.24.4.tar.gz", hash = "sha256:80f5e3a4e498641401868df4208b74581206afbee7cf7b8329daae82676d9463"}, ] +[[package]] +name = "numpy" +version = "1.26.4" +description = "Fundamental package for array computing in Python" +optional = false +python-versions = ">=3.9" +files = [ + {file = "numpy-1.26.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9ff0f4f29c51e2803569d7a51c2304de5554655a60c5d776e35b4a41413830d0"}, + {file = "numpy-1.26.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:2e4ee3380d6de9c9ec04745830fd9e2eccb3e6cf790d39d7b98ffd19b0dd754a"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d209d8969599b27ad20994c8e41936ee0964e6da07478d6c35016bc386b66ad4"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ffa75af20b44f8dba823498024771d5ac50620e6915abac414251bd971b4529f"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:62b8e4b1e28009ef2846b4c7852046736bab361f7aeadeb6a5b89ebec3c7055a"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:a4abb4f9001ad2858e7ac189089c42178fcce737e4169dc61321660f1a96c7d2"}, + {file = "numpy-1.26.4-cp310-cp310-win32.whl", hash = "sha256:bfe25acf8b437eb2a8b2d49d443800a5f18508cd811fea3181723922a8a82b07"}, + {file = "numpy-1.26.4-cp310-cp310-win_amd64.whl", hash = "sha256:b97fe8060236edf3662adfc2c633f56a08ae30560c56310562cb4f95500022d5"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:4c66707fabe114439db9068ee468c26bbdf909cac0fb58686a42a24de1760c71"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:edd8b5fe47dab091176d21bb6de568acdd906d1887a4584a15a9a96a1dca06ef"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7ab55401287bfec946ced39700c053796e7cc0e3acbef09993a9ad2adba6ca6e"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:666dbfb6ec68962c033a450943ded891bed2d54e6755e35e5835d63f4f6931d5"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:96ff0b2ad353d8f990b63294c8986f1ec3cb19d749234014f4e7eb0112ceba5a"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:60dedbb91afcbfdc9bc0b1f3f402804070deed7392c23eb7a7f07fa857868e8a"}, + {file = "numpy-1.26.4-cp311-cp311-win32.whl", hash = "sha256:1af303d6b2210eb850fcf03064d364652b7120803a0b872f5211f5234b399f20"}, + {file = "numpy-1.26.4-cp311-cp311-win_amd64.whl", hash = "sha256:cd25bcecc4974d09257ffcd1f098ee778f7834c3ad767fe5db785be9a4aa9cb2"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:b3ce300f3644fb06443ee2222c2201dd3a89ea6040541412b8fa189341847218"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:03a8c78d01d9781b28a6989f6fa1bb2c4f2d51201cf99d3dd875df6fbd96b23b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9fad7dcb1aac3c7f0584a5a8133e3a43eeb2fe127f47e3632d43d677c66c102b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:675d61ffbfa78604709862923189bad94014bef562cc35cf61d3a07bba02a7ed"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:ab47dbe5cc8210f55aa58e4805fe224dac469cde56b9f731a4c098b91917159a"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:1dda2e7b4ec9dd512f84935c5f126c8bd8b9f2fc001e9f54af255e8c5f16b0e0"}, + {file = "numpy-1.26.4-cp312-cp312-win32.whl", hash = "sha256:50193e430acfc1346175fcbdaa28ffec49947a06918b7b92130744e81e640110"}, + {file = "numpy-1.26.4-cp312-cp312-win_amd64.whl", hash = "sha256:08beddf13648eb95f8d867350f6a018a4be2e5ad54c8d8caed89ebca558b2818"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:7349ab0fa0c429c82442a27a9673fc802ffdb7c7775fad780226cb234965e53c"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:52b8b60467cd7dd1e9ed082188b4e6bb35aa5cdd01777621a1658910745b90be"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d5241e0a80d808d70546c697135da2c613f30e28251ff8307eb72ba696945764"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f870204a840a60da0b12273ef34f7051e98c3b5961b61b0c2c1be6dfd64fbcd3"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:679b0076f67ecc0138fd2ede3a8fd196dddc2ad3254069bcb9faf9a79b1cebcd"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:47711010ad8555514b434df65f7d7b076bb8261df1ca9bb78f53d3b2db02e95c"}, + {file = "numpy-1.26.4-cp39-cp39-win32.whl", hash = "sha256:a354325ee03388678242a4d7ebcd08b5c727033fcff3b2f536aea978e15ee9e6"}, + {file = "numpy-1.26.4-cp39-cp39-win_amd64.whl", hash = "sha256:3373d5d70a5fe74a2c1bb6d2cfd9609ecf686d47a2d7b1d37a8f3b6bf6003aea"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:afedb719a9dcfc7eaf2287b839d8198e06dcd4cb5d276a3df279231138e83d30"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:95a7476c59002f2f6c590b9b7b998306fba6a5aa646b1e22ddfeaf8f78c3a29c"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:7e50d0a0cc3189f9cb0aeb3a6a6af18c16f59f004b866cd2be1c14b36134a4a0"}, + {file = "numpy-1.26.4.tar.gz", hash = "sha256:2a02aba9ed12e4ac4eb3ea9421c420301a0c6460d9830d74a9df87efa4912010"}, +] + [[package]] name = "openai" version = "1.29.0" @@ -1268,4 +1310,4 @@ watchmedo = ["PyYAML (>=3.10)"] [metadata] lock-version = "2.0" python-versions = ">=3.8.1,<4.0" -content-hash = "5c1d82bd3c1c750d2cf8999ca6b8ecc9c0fe96cd348f2aed85e22b7cac54b2de" +content-hash = "40c2e6a7024c39612f27436a3016a336bcf9e0aa7b0468a111ed6220c5190a8d" diff --git a/libs/partners/openai/pyproject.toml b/libs/partners/openai/pyproject.toml index 441d6f10d69f9..aeca614d981cd 100644 --- a/libs/partners/openai/pyproject.toml +++ b/libs/partners/openai/pyproject.toml @@ -29,7 +29,11 @@ pytest-asyncio = "^0.21.1" langchain-core = { path = "../../core", develop = true } pytest-cov = "^4.1.0" langchain-standard-tests = { path = "../../standard-tests", develop = true } -numpy = "^1.24" +# Support Python 3.8 and 3.12+. +numpy = [ + {version = "^1", python = "<3.12"}, + {version = "^1.26.0", python = ">=3.12"} +] [tool.poetry.group.codespell] optional = true @@ -58,7 +62,11 @@ langchain-core = { path = "../../core", develop = true } optional = true [tool.poetry.group.test_integration.dependencies] -numpy = "^1" +# Support Python 3.8 and 3.12+. +numpy = [ + {version = "^1", python = "<3.12"}, + {version = "^1.26.0", python = ">=3.12"} +] [tool.ruff.lint] select = [ diff --git a/libs/partners/pinecone/poetry.lock b/libs/partners/pinecone/poetry.lock index 42b3dbab17469..36a0c4f9d04f1 100644 --- a/libs/partners/pinecone/poetry.lock +++ b/libs/partners/pinecone/poetry.lock @@ -1,4 +1,4 @@ -# This file is automatically @generated by Poetry 1.7.1 and should not be changed by hand. +# This file is automatically @generated by Poetry 1.8.3 and should not be changed by hand. [[package]] name = "annotated-types" @@ -318,7 +318,7 @@ files = [ [[package]] name = "langchain-core" -version = "0.2.0rc1" +version = "0.2.5" description = "Building applications with LLMs through composability" optional = false python-versions = ">=3.8.1,<4.0" @@ -327,22 +327,19 @@ develop = true [package.dependencies] jsonpatch = "^1.33" -langsmith = "^0.1.0" -packaging = "^23.2" +langsmith = "^0.1.75" +packaging = ">=23.2,<25" pydantic = ">=1,<3" PyYAML = ">=5.3" tenacity = "^8.1.0" -[package.extras] -extended-testing = ["jinja2 (>=3,<4)"] - [package.source] type = "directory" url = "../../core" [[package]] name = "langchain-openai" -version = "0.1.7" +version = "0.1.8" description = "An integration package connecting OpenAI and LangChain" optional = false python-versions = ">=3.8.1,<4.0" @@ -350,8 +347,8 @@ files = [] develop = true [package.dependencies] -langchain-core = ">=0.1.46,<0.3" -openai = "^1.24.0" +langchain-core = ">=0.2.2,<0.3" +openai = "^1.26.0" tiktoken = ">=0.7,<1" [package.source] @@ -360,13 +357,13 @@ url = "../openai" [[package]] name = "langsmith" -version = "0.1.59" +version = "0.1.77" description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform." optional = false python-versions = "<4.0,>=3.8.1" files = [ - {file = "langsmith-0.1.59-py3-none-any.whl", hash = "sha256:445e3bc1d3baa1e5340cd979907a19483b9763a2ed37b863a01113d406f69345"}, - {file = "langsmith-0.1.59.tar.gz", hash = "sha256:e748a89f4dd6aa441349143e49e546c03b5dfb43376a25bfef6a5ca792fe1437"}, + {file = "langsmith-0.1.77-py3-none-any.whl", hash = "sha256:2202cc21b1ed7e7b9e5d2af2694be28898afa048c09fdf09f620cbd9301755ae"}, + {file = "langsmith-0.1.77.tar.gz", hash = "sha256:4ace09077a9a4e412afeb4b517ca68e7de7b07f36e4792dc8236ac5207c0c0c7"}, ] [package.dependencies] @@ -472,6 +469,51 @@ files = [ {file = "numpy-1.24.4.tar.gz", hash = "sha256:80f5e3a4e498641401868df4208b74581206afbee7cf7b8329daae82676d9463"}, ] +[[package]] +name = "numpy" +version = "1.26.4" +description = "Fundamental package for array computing in Python" +optional = false +python-versions = ">=3.9" +files = [ + {file = "numpy-1.26.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9ff0f4f29c51e2803569d7a51c2304de5554655a60c5d776e35b4a41413830d0"}, + {file = "numpy-1.26.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:2e4ee3380d6de9c9ec04745830fd9e2eccb3e6cf790d39d7b98ffd19b0dd754a"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d209d8969599b27ad20994c8e41936ee0964e6da07478d6c35016bc386b66ad4"}, + {file = "numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ffa75af20b44f8dba823498024771d5ac50620e6915abac414251bd971b4529f"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:62b8e4b1e28009ef2846b4c7852046736bab361f7aeadeb6a5b89ebec3c7055a"}, + {file = "numpy-1.26.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:a4abb4f9001ad2858e7ac189089c42178fcce737e4169dc61321660f1a96c7d2"}, + {file = "numpy-1.26.4-cp310-cp310-win32.whl", hash = "sha256:bfe25acf8b437eb2a8b2d49d443800a5f18508cd811fea3181723922a8a82b07"}, + {file = "numpy-1.26.4-cp310-cp310-win_amd64.whl", hash = "sha256:b97fe8060236edf3662adfc2c633f56a08ae30560c56310562cb4f95500022d5"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:4c66707fabe114439db9068ee468c26bbdf909cac0fb58686a42a24de1760c71"}, + {file = "numpy-1.26.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:edd8b5fe47dab091176d21bb6de568acdd906d1887a4584a15a9a96a1dca06ef"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7ab55401287bfec946ced39700c053796e7cc0e3acbef09993a9ad2adba6ca6e"}, + {file = "numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:666dbfb6ec68962c033a450943ded891bed2d54e6755e35e5835d63f4f6931d5"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:96ff0b2ad353d8f990b63294c8986f1ec3cb19d749234014f4e7eb0112ceba5a"}, + {file = "numpy-1.26.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:60dedbb91afcbfdc9bc0b1f3f402804070deed7392c23eb7a7f07fa857868e8a"}, + {file = "numpy-1.26.4-cp311-cp311-win32.whl", hash = "sha256:1af303d6b2210eb850fcf03064d364652b7120803a0b872f5211f5234b399f20"}, + {file = "numpy-1.26.4-cp311-cp311-win_amd64.whl", hash = "sha256:cd25bcecc4974d09257ffcd1f098ee778f7834c3ad767fe5db785be9a4aa9cb2"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:b3ce300f3644fb06443ee2222c2201dd3a89ea6040541412b8fa189341847218"}, + {file = "numpy-1.26.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:03a8c78d01d9781b28a6989f6fa1bb2c4f2d51201cf99d3dd875df6fbd96b23b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9fad7dcb1aac3c7f0584a5a8133e3a43eeb2fe127f47e3632d43d677c66c102b"}, + {file = "numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:675d61ffbfa78604709862923189bad94014bef562cc35cf61d3a07bba02a7ed"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:ab47dbe5cc8210f55aa58e4805fe224dac469cde56b9f731a4c098b91917159a"}, + {file = "numpy-1.26.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:1dda2e7b4ec9dd512f84935c5f126c8bd8b9f2fc001e9f54af255e8c5f16b0e0"}, + {file = "numpy-1.26.4-cp312-cp312-win32.whl", hash = "sha256:50193e430acfc1346175fcbdaa28ffec49947a06918b7b92130744e81e640110"}, + {file = "numpy-1.26.4-cp312-cp312-win_amd64.whl", hash = "sha256:08beddf13648eb95f8d867350f6a018a4be2e5ad54c8d8caed89ebca558b2818"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:7349ab0fa0c429c82442a27a9673fc802ffdb7c7775fad780226cb234965e53c"}, + {file = "numpy-1.26.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:52b8b60467cd7dd1e9ed082188b4e6bb35aa5cdd01777621a1658910745b90be"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d5241e0a80d808d70546c697135da2c613f30e28251ff8307eb72ba696945764"}, + {file = "numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f870204a840a60da0b12273ef34f7051e98c3b5961b61b0c2c1be6dfd64fbcd3"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:679b0076f67ecc0138fd2ede3a8fd196dddc2ad3254069bcb9faf9a79b1cebcd"}, + {file = "numpy-1.26.4-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:47711010ad8555514b434df65f7d7b076bb8261df1ca9bb78f53d3b2db02e95c"}, + {file = "numpy-1.26.4-cp39-cp39-win32.whl", hash = "sha256:a354325ee03388678242a4d7ebcd08b5c727033fcff3b2f536aea978e15ee9e6"}, + {file = "numpy-1.26.4-cp39-cp39-win_amd64.whl", hash = "sha256:3373d5d70a5fe74a2c1bb6d2cfd9609ecf686d47a2d7b1d37a8f3b6bf6003aea"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:afedb719a9dcfc7eaf2287b839d8198e06dcd4cb5d276a3df279231138e83d30"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:95a7476c59002f2f6c590b9b7b998306fba6a5aa646b1e22ddfeaf8f78c3a29c"}, + {file = "numpy-1.26.4-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:7e50d0a0cc3189f9cb0aeb3a6a6af18c16f59f004b866cd2be1c14b36134a4a0"}, + {file = "numpy-1.26.4.tar.gz", hash = "sha256:2a02aba9ed12e4ac4eb3ea9421c420301a0c6460d9830d74a9df87efa4912010"}, +] + [[package]] name = "openai" version = "1.30.1" @@ -1196,4 +1238,4 @@ watchmedo = ["PyYAML (>=3.10)"] [metadata] lock-version = "2.0" python-versions = ">=3.8.1,<3.13" -content-hash = "56e1802b64dbffe94b4400814e20dd9754d5b9b5a4f1b28ace3ed3d17a252502" +content-hash = "95a680ed7442117495756dc115f0b73c513f9cb516376ea978040fd18e298811" diff --git a/libs/partners/pinecone/pyproject.toml b/libs/partners/pinecone/pyproject.toml index ea10cfcec33d0..d38e703bbf7c2 100644 --- a/libs/partners/pinecone/pyproject.toml +++ b/libs/partners/pinecone/pyproject.toml @@ -15,7 +15,11 @@ license = "MIT" python = ">=3.8.1,<3.13" langchain-core = ">=0.1.52,<0.3" pinecone-client = "^3.2.2" -numpy = "^1" +# Support Python 3.8 and 3.12+. +numpy = [ + {version = "^1", python = "<3.12"}, + {version = "^1.26.0", python = ">=3.12"} +] [tool.poetry.group.test] optional = true