Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task Stuck in Submitted Status with Invalid Partition ID #811

Open
qdelamea-aneo opened this issue Dec 18, 2024 · 0 comments
Open

Task Stuck in Submitted Status with Invalid Partition ID #811

qdelamea-aneo opened this issue Dec 18, 2024 · 0 comments

Comments

@qdelamea-aneo
Copy link

Description

When submitting a task with a partition_id that does not exist, the task stays indefinitely in the Submitted status instead of failing or being rejected with an appropriate error message.


Steps to Reproduce

  1. Use commit a0ed39c6 from the ArmoniK repository.

  2. Set-up a virtual environment and install the package armonik with version 3.22.0.

  3. Run the following Python script to reproduce the issue:

    import time
    
    from datetime import timedelta
    
    from armonik.client import ArmoniKTasks, ArmoniKResults, ArmoniKSessions
    from armonik.common import Task, TaskOptions, TaskDefinition, TaskStatus
    from armonik.common.channel import create_channel
    
    
    with create_channel("http://172.17.119.85:5001") as channel:
        session_client = ArmoniKSessions(channel)
        result_client = ArmoniKResults(channel)
        task_client = ArmoniKTasks(channel)
    
        session_id = session_client.create_session(
            default_task_options=TaskOptions(
                max_retries=1,
                max_duration=timedelta(minutes=5),
                priority=1,
            ),
            partition_ids=["default"],
        )
        print(f"Session ID: {session_id}.")
        payload = result_client.create_results(results_data={"payload": "test".encode("utf-8")}, session_id=session_id)["payload"]
        result = result_client.create_results_metadata(result_names=["result"], session_id=session_id)["result"]
        task = task_client.submit_tasks(
            session_id=session_id,
            tasks=[TaskDefinition(
                payload_id=payload.result_id,
                expected_output_ids=result.result_id,
                options=TaskOptions(
                    max_retries=1,
                    max_duration=timedelta(minutes=5),
                    priority=1,
                    partition_id="wrong",
                ),
            )]
        )[0]
        elapsed = 0
        try:
            while task.status != TaskStatus.COMPLETED:
                task.refresh(task_client=task_client)
                print(f"Task {task.id} status is {TaskStatus(task.status).name} [{elapsed}s elapsed].")
                time.sleep(1.0)
                elapsed += 1
        except Exception as error:
            print(error)
            task.refresh(task_client=task_client)
            print(f"Task {task.id} status is {TaskStatus(task.status).name} after failure.")
  4. Observe that the task's status remains Submitted indefinitely, even though the partition_id is invalid.


Expected Behavior

The system should validate the partition_id when a task is submitted. If the partition_id is invalid, the task should:

  • Fail immediately, returning an appropriate error message, or
  • Transition to an error state that can be retrieved programmatically.

Additional Information

  • ArmoniK Repo Commit: a0ed39c6
  • System: Ubuntu 22.04.5 LTS.
  • Logs/Errors: None available for this issue in the script provided, as no explicit error is raised.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant