Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

316-Fix-BasicCarMaintenanceUITests #348

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

316-Fix-BasicCarMaintenanceUITests #348

wants to merge 1 commit into from

Conversation

lexbonder
Copy link

What it Does

How I Tested

  • Open file BasicCarMaintenanceUITests
  • Click play button on line 20 func testAddVehicle() {
  • See success popup in Xcode

Notes

  • This is my first ever open source contribution, and first time working with XCTest, so naturally I have many questions:

  • The error I was getting said that it was failing to scroll. I thought it may have something to do with the Thanks for using this app popup so I found a way to dismiss it.

  • Dismissing the popup allowed the test to pass. But the test isn't actually asserting anything. Is that right for a UI test? A successful tap of the Add button is a successful test? Should I be asserting that a new vehicle has been added?

  • Is the pattern I used for the Extension folder and file inside good? is there a better pattern that Should be used?

  • It looks like I set up firebase correctly for when I run the app, but the data doesn't go away each time I run the tests, have I missed a step or is that working as expected?

  • Should I edit the test to fill in the rest of the inputs?

@lexbonder
316-Fix-BasicCarMaintenanceUITests
5f74e28 
This is my first ever open source contribution, and first time working with XCTest, so naturally I have many questions:

The error I was getting said that it was failing to scroll. I thought it may have something to do with the 'Thanks for using this app' popup so I found a way to dismiss it.

Dismissing the popup allowed the test to pass. But the test isn't actually asserting anything. Is that right for a UI test? A successful tap of the 'add' button is a successful test? Should I be asserting that a new vehicle has been added?

It looks like I set up firebase correctly for when I run the app, but the data doesn't go away each time I run the tests, have I missed a step or is that working as expected?

Should I edit the test to fill in the rest of the inputs?

Thanks
@mikaelacaron
Copy link
Owner

The error I was getting said that it was failing to scroll. I thought it may have something to do with the Thanks for using this app popup so I found a way to dismiss it.

sounds like a good solution, does this work though when the test is run multiple times? Like the TipKit popup shows up every time the UI test is run?

Dismissing the popup allowed the test to pass. But the test isn't actually asserting anything. Is that right for a UI test? A successful tap of the Add button is a successful test? Should I be asserting that a new vehicle has been added?

Technically yes, but because this codebase is still pretty new, the testing coverage and functionality isn't the best, that's something I want to improve over time or possibly after Hacktoberfest. Also one component of this, is if the Firebase emulator is being set up correctly for running UI or unit tests. Like it's set up for development, but is that the same setup for testing? (I'm not sure)

Is the pattern I used for the Extension folder and file inside good? is there a better pattern that Should be used?

yes that's fine!

It looks like I set up firebase correctly for when I run the app, but the data doesn't go away each time I run the tests, have I missed a step or is that working as expected?

This is because of how Firebase emulator is used with this command: firebase emulators:start --import=./local-data --export-on-exit

if you didn't use --import=./local-data --export-on-exit the data would go away each time

Should I edit the test to fill in the rest of the inputs?

Not in this PR, but make a new issue for this! (which will probably also include checking that all the data was added correctly

I'd love in the future to have all the UI tests be used to create localized screenshots, and have those run automatically with Fastlane! that would be really cool, but I think we're still a little ways away from that

mikaelacaron
mikaelacaron requested changes Oct 27, 2024
View reviewed changes
Copy link
Owner

@mikaelacaron mikaelacaron left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! I tried to run the UI tests, but they didn't pass on my machine, I dropped the error below

once you fix it, please click resolve, and then re-review and I'll take another look!
Re-review button

Basic-Car-Maintenance-UITests/Extensions/XCUIApplication-dismissPopup.swift

extension XCUIApplication {
func dismissPopup() {
otherElements["dismiss popup"].tap()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change this function to dismissTipKitPopup

but before that, question, does this only work for TipKit, or what other "popups" would this function be used for? if it's specific to this test, I would actually move it so it's in the BasicCarMaintenanceUITests class

Basic-Car-Maintenance-UITests/BasicCarMaintenanceUITests.swift
@@ -18,6 +18,8 @@ final class BasicCarMaintenanceUITests: XCTestCase {
}

func testAddVehicle() {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't pass for me, I get this error.
I erased the simulator, and deleted all the data from the emulator

Running tests...
Failed to send CA Event for app launch measurements for ca_event_type: 0 event_name: com.apple.app_launch_measurement.FirstFramePresentationMetric
Failed to send CA Event for app launch measurements for ca_event_type: 1 event_name: com.apple.app_launch_measurement.ExtendedLaunchMetrics
    t =      nans Interface orientation changed to Portrait
Test Suite 'BasicCarMaintenanceUITests' started at 2024-10-27 16:26:30.306.
Test Case '-[Basic_Car_Maintenance_UITests.BasicCarMaintenanceUITests testAddVehicle]' started.
    t =     0.00s Start Test at 2024-10-27 16:26:30.307
    t =     0.13s Set Up
    t =     0.13s     Open com.icyappstudio.Basic-Car-Maintenance
    t =     0.13s         Launch com.icyappstudio.Basic-Car-Maintenanc
warning: (arm64) /Users/mikaelacaron/Library/Developer/Xcode/DerivedData/Basic-Car-Maintenance-hasiwwufjtgocfeyrzxbvwqlinjv/Build/Products/Debug-iphonesimulator/Basic-Car-Maintenance.app/Basic-Car-Maintenance empty dSYM file detected, dSYM was created with an executable with no debug info.
e
    t =     1.58s             Setting up automation session
    t =     2.48s             Wait for com.icyappstudio.Basic-Car-Maintenance to idle
    t =     3.97s Tap "dismiss popup" Other
    t =     3.97s     Wait for com.icyappstudio.Basic-Car-Maintenance to idle
    t =     4.35s     Find the "dismiss popup" Other
    t =     5.47s         Find the "dismiss popup" Other (retry 1)
    t =     6.55s         Find the "dismiss popup" Other (retry 2)
    t =     6.61s Requesting snapshot of accessibility hierarchy for app with pid 7176
/Users/mikaelacaron/Documents/Apps/Basic-Car-Maintenance/Basic-Car-Maintenance-UITests/Extensions/XCUIApplication-dismissPopup.swift:13: error: -[Basic_Car_Maintenance_UITests.BasicCarMaintenanceUITests testAddVehicle] : Failed to tap "dismiss popup" Other: No matches found for Elements matching predicate '"dismiss popup" IN identifiers' from input {(
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other, label: 'Dismiss',
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other, label: 'Filter',
    Other, identifier: 'plus', label: 'Add',
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other,
    Other, label: 'Vertical scroll bar, 1 page', value: 0%,
    Other
)}
Possibly caused by runtime issues:
Automation type mismatch: computed Other from legacy attributes vs ActivityIndicator from modern attribute. Input attributes and values: {
    "XC_kAXXCAttributeAutomationType" = 36;
    "XC_kAXXCAttributeElementBaseType" = UIResponder;
    "XC_kAXXCAttributeElementType" = "SwiftUI.AccessibilityNode";
    "XC_kAXXCAttributeTraits" = 8589935104;
}
Automation type mismatch: computed Other from legacy attributes vs StaticText from modern attribute. Input attributes and values: {
    "XC_kAXXCAttributeAutomationType" = 48;
    "XC_kAXXCAttributeElementBaseType" = UILabel;
    "XC_kAXXCAttributeElementType" = UILabel;
    "XC_kAXXCAttributeTraits" = 8590000128;
}
See test report attachments for more detail.
    t =     6.83s Tear Down
Test Case '-[Basic_Car_Maintenance_UITests.BasicCarMaintenanceUITests testAddVehicle]' failed (7.091 seconds).
Test Suite 'BasicCarMaintenanceUITests' failed at 2024-10-27 16:26:37.398.
	 Executed 1 test, with 1 failure (0 unexpected) in 7.091 (7.092) seconds

@mikaelacaron mikaelacaron added the hacktoberfest-accepted Accepted PR for Hacktoberfest label Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mikaelacaron mikaelacaron mikaelacaron requested changes

Assignees
No one assigned
Labels
hacktoberfest-accepted Accepted PR for Hacktoberfest
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Fix BasicCarMaintenanceUITests testAddVehicle Fails
2 participants
@lexbonder @mikaelacaron