Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grand dispatch queue #661

Open
wants to merge 169 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 168 commits
Commits
Show all changes
169 commits
Select commit Hold shift + click to select a range
9389f14
CMakeLists.txt
sbSteveK Jul 29, 2024
e825924
event_loop.h
sbSteveK Jul 29, 2024
452217c
io.h
sbSteveK Jul 29, 2024
c0d4086
kqueue_event_loop.c
sbSteveK Jul 29, 2024
84fd773
dispatch_queue_event_loop.c
sbSteveK Jul 29, 2024
98c8ef0
event_loop.c
sbSteveK Jul 29, 2024
89e8ece
epoll_event_loop.c
sbSteveK Jul 29, 2024
9c27144
iocp_event_loop.c
sbSteveK Jul 29, 2024
f375bb2
event_loop_test.c
sbSteveK Jul 29, 2024
41a5fa1
try ifdef on TargetConditionals.h and comment failing event loop test
sbSteveK Jul 29, 2024
04c2b93
reorder the event loop config, so apple platform would use dispatch q…
xiazhvera Jul 30, 2024
0d301d2
bring in dispatch queue changes
xiazhvera Sep 3, 2024
4afaea6
clangformat
xiazhvera Sep 3, 2024
a7f6904
remove unused args
xiazhvera Sep 3, 2024
89635db
clean up
xiazhvera Sep 11, 2024
195ca1c
clean up dispatch queue
xiazhvera Sep 11, 2024
24ce65b
Merge branch 'main' of https://github.com/awslabs/aws-c-io into grand…
xiazhvera Sep 11, 2024
287094f
clang-format
xiazhvera Sep 11, 2024
bd58da0
more comments and format clean up
xiazhvera Sep 11, 2024
f0e5dde
quick windows test
xiazhvera Sep 16, 2024
aef1b14
TEST: quick error verification
xiazhvera Sep 16, 2024
41bb257
Revert "TEST: quick error verification"
xiazhvera Sep 16, 2024
22e68b2
TEST: using struct instead of union
xiazhvera Sep 16, 2024
a28cb37
Revert "TEST: using struct instead of union"
xiazhvera Sep 16, 2024
c67e966
add back definition for union
xiazhvera Sep 16, 2024
3ca34ce
WINDOWS: rename function
xiazhvera Sep 16, 2024
f8c26f5
fix compile error
xiazhvera Sep 16, 2024
a428cd8
remove unused finalize functions
xiazhvera Sep 16, 2024
5ab8f24
fix event loop schedule future
xiazhvera Sep 16, 2024
0918e76
improve dispatch caller's thread check
xiazhvera Sep 26, 2024
a55f14f
update caller's thread changes
xiazhvera Sep 26, 2024
06fb206
use lock to protect the thread id info
xiazhvera Sep 26, 2024
ed04764
lint
xiazhvera Sep 26, 2024
e8fe46d
fix thread related test/disable pipe tests
xiazhvera Sep 26, 2024
a84cb5a
AWS_USE_DISPATCH_QUEUE updates (#679)
sbSteveK Oct 3, 2024
977bb8a
Merge branch 'main' into grand_dispatch_queue
xiazhvera Oct 15, 2024
ce07c5a
bring in event loop changes
xiazhvera Oct 15, 2024
0210346
bring in CI changes
xiazhvera Oct 15, 2024
a751d76
Merge branch 'main' into grand_dispatch_queue
xiazhvera Oct 16, 2024
b44c510
update comments
xiazhvera Oct 25, 2024
b0f85f2
remove is_executing check
xiazhvera Oct 25, 2024
7bc39ee
improve comments
xiazhvera Oct 25, 2024
475c1f2
make all private function static
xiazhvera Oct 25, 2024
cf592a7
init variables
xiazhvera Oct 25, 2024
1803c0f
clang-format
xiazhvera Oct 25, 2024
9d97302
Checkpoint
bretambrose Oct 30, 2024
9781845
Updated with doc comments
Oct 31, 2024
754c56d
Creation API
Oct 31, 2024
974a9b2
Checkpoint
Nov 5, 2024
0b7b061
Merge branch 'main' into EventLoopPublicApi
Nov 5, 2024
75e4f45
Formatting
Nov 5, 2024
4a784cc
Oops
Nov 5, 2024
381f3dd
Windows updates
Nov 5, 2024
e63c0a8
test update
Nov 5, 2024
09cff00
[WIP] API update for runtime switch event loop
xiazhvera Nov 7, 2024
ca3a134
update event loop group creation
xiazhvera Nov 7, 2024
6619695
clang format
xiazhvera Nov 7, 2024
7a89d9e
revert shutdown_types?
xiazhvera Nov 7, 2024
56fa4d1
rename cmake flags
xiazhvera Nov 7, 2024
c1a4971
fix default event loop
xiazhvera Nov 7, 2024
470164b
improve error message
xiazhvera Nov 7, 2024
977c4f7
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 7, 2024
9e6d574
Update based on PR feedback:
Nov 7, 2024
eab14fa
fix header update
xiazhvera Nov 7, 2024
d8f5fdd
Merge branch 'EventLoopPublicApi' of github.com:awslabs/aws-c-io into…
xiazhvera Nov 7, 2024
9323cc3
Make io testing channel usable across library boundaries
Nov 7, 2024
b771d8c
Revert "Make io testing channel usable across library boundaries"
Nov 7, 2024
4f3048e
add function to override the default event loop type
xiazhvera Nov 7, 2024
fcb3d9a
Revert "Revert "Make io testing channel usable across library boundar…
Nov 8, 2024
9b16a3b
Set allocator
Nov 8, 2024
a30cd39
default event loop type override
xiazhvera Nov 8, 2024
74733ad
hide the test help function as internal private
xiazhvera Nov 8, 2024
558d179
clang format
xiazhvera Nov 8, 2024
16d4e25
remove unreachable..
xiazhvera Nov 8, 2024
55de953
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 8, 2024
74019cf
update ci flags
xiazhvera Nov 8, 2024
5d22a69
update setup switch default event loop
xiazhvera Nov 8, 2024
600421e
revert function rename
xiazhvera Nov 8, 2024
7cb0936
update cmake for dispatch queue
xiazhvera Nov 8, 2024
f808b35
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 8, 2024
e84a1a6
introduce socket vtable
xiazhvera Nov 8, 2024
d79b5b7
fix platform error code
xiazhvera Nov 8, 2024
3ce216c
remove global vtable
xiazhvera Nov 9, 2024
6233c9d
fix flag
xiazhvera Nov 9, 2024
4a8007a
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 9, 2024
70008b1
fix implicit function call
xiazhvera Nov 9, 2024
0f75185
set apple networkframework flag
xiazhvera Nov 9, 2024
f7fb5a2
prevent fail fast
xiazhvera Nov 9, 2024
8bd9808
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 9, 2024
16861b9
update cmake event loop defines
xiazhvera Nov 9, 2024
8d946db
temporary remove dispatch queue wrap
xiazhvera Nov 9, 2024
17c7cca
temporary remove dispatch queue wrap
xiazhvera Nov 9, 2024
1212977
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 9, 2024
afd634d
update dispatch queue related flags
xiazhvera Nov 9, 2024
d68acdb
Doc comments
bretambrose Nov 11, 2024
6b92e59
hide dispatch queue header
xiazhvera Nov 11, 2024
3999196
make apple network framework public
xiazhvera Nov 11, 2024
6f1984c
Merge branch 'EventLoopPublicApi' of github.com:awslabs/aws-c-io into…
xiazhvera Nov 11, 2024
8d84d11
Apply suggestions from code review
xiazhvera Nov 11, 2024
71fae6f
update cr changes
xiazhvera Nov 11, 2024
35efc2e
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 11, 2024
405c988
rename enum
xiazhvera Nov 12, 2024
e614bdc
Merge branch 'main' of github.com:awslabs/aws-c-io into runtime_selec…
xiazhvera Nov 12, 2024
e22131a
Merge branch 'main' into runtime_select_event_loop
xiazhvera Nov 12, 2024
51e2d5a
eliminate event loop constructor
xiazhvera Nov 12, 2024
8be6cd2
clean up and format
xiazhvera Nov 12, 2024
94f95c1
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
ab05a35
Merge branch 'main' into runtime_select_event_loop
xiazhvera Nov 12, 2024
61cbc90
lint
xiazhvera Nov 12, 2024
4d0e12c
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
22c6363
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
c507d13
update comments
xiazhvera Nov 12, 2024
c54b99e
rename enum
xiazhvera Nov 12, 2024
69cbb09
lint
xiazhvera Nov 12, 2024
7b51b56
wrap the kqueue function
xiazhvera Nov 12, 2024
5fc5270
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
aa876a1
add posix file for non-darwin Apple platform
xiazhvera Nov 12, 2024
ee7fa76
fix lint
xiazhvera Nov 12, 2024
7d20796
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 12, 2024
2fb32ab
handling library error in cmake
xiazhvera Nov 12, 2024
f5a5338
Merge branch 'main' into runtime_select_event_loop
xiazhvera Nov 13, 2024
7ea8588
renmae event loop new function
xiazhvera Nov 13, 2024
1cbe989
update code review comments
xiazhvera Nov 21, 2024
667e41a
add unit test
xiazhvera Nov 22, 2024
48ad48c
move to private socket header
xiazhvera Nov 22, 2024
17b79a4
move function definition
xiazhvera Nov 22, 2024
a32ee15
include private header & rename function
xiazhvera Nov 22, 2024
c53b4ad
include private socket header
xiazhvera Nov 22, 2024
ad5152c
format
xiazhvera Nov 22, 2024
1afb859
move windows related header to private
xiazhvera Nov 25, 2024
691aa17
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 25, 2024
182757f
fix unreferenced param
xiazhvera Nov 25, 2024
02afc29
rename windows creation
xiazhvera Nov 25, 2024
6610f79
format
xiazhvera Nov 25, 2024
53fc1fc
add event loop creation test for windows
xiazhvera Nov 25, 2024
2461b43
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Nov 25, 2024
6783915
Grand dispatch queue context (#697)
xiazhvera Nov 29, 2024
6fcd019
Merge branch 'main' into runtime_select_event_loop
xiazhvera Dec 3, 2024
85bf6ce
move aws_io_handle
xiazhvera Dec 3, 2024
ef012d3
schedule service entry on dispatch queue resume
xiazhvera Dec 3, 2024
31a9a95
update cr
xiazhvera Dec 4, 2024
8bb613e
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Dec 4, 2024
7cdd319
WIP code review update
xiazhvera Dec 5, 2024
e1d7513
remove apple network framewokr CI so that we dont block merge
xiazhvera Dec 9, 2024
d45eb98
wip update cr
xiazhvera Dec 10, 2024
98c558e
update vcc and related hash
xiazhvera Dec 17, 2024
c8e57c5
update checksum
xiazhvera Dec 17, 2024
cfa0399
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Dec 18, 2024
f4414aa
refactor dispatch queue release process/update iteartion schedule pro…
xiazhvera Dec 18, 2024
e1ce086
set up impl_data of event loop ealier
xiazhvera Dec 18, 2024
a93216b
revert ci change
xiazhvera Dec 18, 2024
1cc1633
Merge branch 'main' into runtime_select_event_loop
xiazhvera Jan 6, 2025
7adfffb
update code review
xiazhvera Jan 7, 2025
485e376
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Jan 7, 2025
5e1e728
Merge branch 'main' into runtime_select_event_loop
xiazhvera Jan 7, 2025
893f94b
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Jan 7, 2025
18b5d82
fix CI flag
xiazhvera Jan 7, 2025
f9ff79a
lint and fix warning
xiazhvera Jan 7, 2025
98ea949
Merge branch 'runtime_select_event_loop' of github.com:awslabs/aws-c-…
xiazhvera Jan 7, 2025
6933da0
rename private headers
xiazhvera Jan 7, 2025
16c36e8
cr update
xiazhvera Jan 7, 2025
d986649
improve comments and lock results
xiazhvera Jan 8, 2025
54f4883
Merge branch 'main' of github.com:awslabs/aws-c-io into grand_dispatc…
xiazhvera Jan 8, 2025
84c6c4c
use aws_min & enable sanitizer in ci
xiazhvera Jan 8, 2025
cce6210
use priority queue for service entry list
xiazhvera Jan 9, 2025
7058676
clean up priority_queue
xiazhvera Jan 9, 2025
52fe388
acquire context for iteration
xiazhvera Jan 9, 2025
c2dab8d
clean up comments
xiazhvera Jan 9, 2025
4fadfee
fix memory leak, and fix dispatch_queue_id_prefix
xiazhvera Jan 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,8 @@ jobs:
strategy:
fail-fast: false
matrix:
eventloop: ["kqueue"] # TODO: Add "dispatch_queue" when apple network framework is implemented.
eventloop: ["kqueue", "dispatch_queue"]
sanitizers: [",thread", ",address,undefined"]
steps:
- uses: aws-actions/configure-aws-credentials@v4
with:
Expand All @@ -242,7 +243,7 @@ jobs:
run: |
python3 -c "from urllib.request import urlretrieve; urlretrieve('${{ env.BUILDER_HOST }}/${{ env.BUILDER_SOURCE }}/${{ env.BUILDER_VERSION }}/builder.pyz?run=${{ env.RUN }}', 'builder')"
chmod a+x builder
./builder build -p ${{ env.PACKAGE_NAME }} --cmake-extra=-DAWS_USE_APPLE_NETWORK_FRAMEWORK=${{ matrix.eventloop == 'dispatch_queue' && 'ON' || 'OFF' }}
./builder build -p ${{ env.PACKAGE_NAME }} --cmake-extra=-DAWS_USE_APPLE_NETWORK_FRAMEWORK=${{ matrix.eventloop == 'dispatch_queue' && 'ON' || 'OFF' }} --cmake-extra=-DENABLE_SANITIZERS=ON --cmake-extra=-DSANITIZERS="${{ matrix.sanitizers }}"

macos-x64:
runs-on: macos-14-large # latest
Expand All @@ -262,7 +263,8 @@ jobs:
strategy:
fail-fast: false
matrix:
eventloop: ["kqueue"] # TODO: Add "-DAWS_USE_APPLE_NETWORK_FRAMEWORK=ON" when apple network framework is implemented.
eventloop: ["kqueue", "dispatch_queue"]
sanitizers: [",thread", ",address,undefined"]
steps:
- uses: aws-actions/configure-aws-credentials@v4
with:
Expand All @@ -272,7 +274,7 @@ jobs:
run: |
python3 -c "from urllib.request import urlretrieve; urlretrieve('${{ env.BUILDER_HOST }}/${{ env.BUILDER_SOURCE }}/${{ env.BUILDER_VERSION }}/builder.pyz?run=${{ env.RUN }}', 'builder')"
chmod a+x builder
./builder build -p ${{ env.PACKAGE_NAME }} --cmake-extra=-DAWS_USE_APPLE_NETWORK_FRAMEWORK=${{ matrix.eventloop == 'dispatch_queue' && 'ON' || 'OFF' }} --config Debug
./builder build -p ${{ env.PACKAGE_NAME }} --cmake-extra=-DAWS_USE_APPLE_NETWORK_FRAMEWORK=${{ matrix.eventloop == 'dispatch_queue' && 'ON' || 'OFF' }} --cmake-extra=-DENABLE_SANITIZERS=ON --cmake-extra=-DSANITIZERS="${{ matrix.sanitizers }}" --config Debug

freebsd:
runs-on: ubuntu-24.04 # latest
Expand Down
1 change: 0 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ file(GLOB AWS_IO_TESTING_HEADERS
"include/aws/testing/*.h"
)


file(GLOB AWS_IO_PRIV_HEADERS
"include/aws/io/private/*.h"
)
Expand Down
7 changes: 7 additions & 0 deletions include/aws/io/io.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,19 @@ AWS_PUSH_SANE_WARNING_LEVEL

#define AWS_C_IO_PACKAGE_ID 1

struct aws_io_handle;
xiazhvera marked this conversation as resolved.
Show resolved Hide resolved
typedef void aws_io_set_queue_on_handle_fn(struct aws_io_handle *handle, void *queue);
typedef void aws_io_clear_queue_on_handle_fn(struct aws_io_handle *handle);

struct aws_io_handle {
union {
int fd;
/* on Apple systems, handle is of type nw_connection_t. On Windows, it's a SOCKET handle. */
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarity needed. I don't think we're using nw_connection_t unless dispatch queue is turned on.

"On Apple systems using dispatch queue event loops handle is of type nw_connection_t"

void *handle;
} data;
void *additional_data;
aws_io_set_queue_on_handle_fn *set_queue;
aws_io_clear_queue_on_handle_fn *clear_queue;
};

enum aws_io_message_type {
Expand Down
10 changes: 10 additions & 0 deletions include/aws/io/private/event_loop_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,16 @@ AWS_IO_API
struct _OVERLAPPED *aws_overlapped_to_windows_overlapped(struct aws_overlapped *overlapped);
#endif /* AWS_ENABLE_IO_COMPLETION_PORTS */

/**
* @internal - Don't use outside of testing.
*
* Return the default event loop type. If the return value is `AWS_ELT_PLATFORM_DEFAULT`, the function failed to
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you changed this to AWS_EVENT_LOOP_PLATFORM_DEFAULT as AWS_ELT_PLATFORM_DEFAULT doesn't exist.

Also, it looks like there is nothing returned on a failure to retrieve the default type value. I think its a compile time error?

* retrieve the default type value.
* If `aws_event_loop_override_default_type` has been called, return the override default type.
*/
AWS_IO_API
enum aws_event_loop_type aws_event_loop_get_default_type(void);

/**
* Associates an aws_io_handle with the event loop's I/O Completion Port.
*
Expand Down
Loading
Loading