Skip to content

Commit

Permalink
Merge pull request #529 from magicyuan876/main
Browse files Browse the repository at this point in the history
修复断点续传bug,过滤已经存在的文档是只过滤状态是已经处理完成的
  • Loading branch information
LarFii authored Dec 30, 2024
2 parents 889d056 + 7271ac6 commit af893a1
Showing 1 changed file with 8 additions and 2 deletions.
10 changes: 8 additions & 2 deletions lightrag/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -341,8 +341,14 @@ def __post_init__(self):
logger.info(f"Loaded document status storage with {len(self._data)} records")

async def filter_keys(self, data: list[str]) -> set[str]:
"""Return keys that don't exist in storage"""
return set([k for k in data if k not in self._data])
"""Return keys that should be processed (not in storage or not successfully processed)"""
return set(
[
k
for k in data
if k not in self._data or self._data[k]["status"] != DocStatus.PROCESSED
]
)

async def get_status_counts(self) -> Dict[str, int]:
"""Get counts of documents in each status"""
Expand Down

0 comments on commit af893a1

Please sign in to comment.