Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Make sure ios tests fail if they fail #20518

Merged
merged 2 commits into from
Aug 14, 2020
Merged

Make sure ios tests fail if they fail #20518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4fcb481
Make sure ios tests fail if they fail
dnfield Aug 14, 2020
d35a058
explain
dnfield Aug 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions testing/ios/IosUnitTests/run_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,6 @@ if [ $# -eq 1 ]; then
FLUTTER_ENGINE=$1
fi

set -o pipefail && xcodebuild -sdk iphonesimulator \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set -o pipefail was the magic that worked with xcpretty. When we removed xcpretty it was no longer magical.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @jmagman

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't being used in the python script, and I couldn't seem to get it working in a way the script liked. Honestly, I'm not sure its worth it. Every time I've ever had a test failure I had to re-run my tests without xcpretty to figure out why.

(@xster same reply to your comment above)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine removing xcpretty, just wanted to explain why this was there and how this broke.

-scheme IosUnitTests \
-destination 'platform=iOS Simulator,name=iPhone 8' \
test \
FLUTTER_ENGINE=$FLUTTER_ENGINE
../../run_tests.py --variant $FLUTTER_ENGINE --type objc --ios-variant $FLUTTER_ENGINE

popd
8 changes: 5 additions & 3 deletions testing/run_tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,17 +370,19 @@ def RunObjcTests(ios_variant='ios_debug_sim_unopt'):
ios_out_dir = os.path.join(out_dir, ios_variant)
EnsureIosTestsAreBuilt(ios_out_dir)

pretty = "cat" if subprocess.call(["which", "xcpretty"]) else "xcpretty"
ios_unit_test_dir = os.path.join(buildroot_dir, 'flutter', 'testing', 'ios', 'IosUnitTests')

# Avoid using xcpretty unless the following can be addressed:
# - Make sure all relevant failure output is printed on a failure.
# - Make sure that a failing exit code is set for CI.
# See https://github.com/flutter/flutter/issues/63742
Comment on lines +375 to +378
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we even mention this? I never want to hide the full Xcode build results when there's an error, it's frustrating when I can't reproduce the failure locally. xcpretty is nice for end users, not me who wants to see all the clang flags and search paths. "relevant failure output" is pretty different depending on what's failing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that thinking that a year or two from now, someone is going to say "Hey, we should add xcpretty to make this easier to read!" again :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it'll probably be you or me. 😄

command = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This removes it locally right? Can we just set -o pipefail && here?

'xcodebuild '
'-sdk iphonesimulator '
'-scheme IosUnitTests '
"-destination platform='iOS Simulator,name=iPhone 8' "
'test '
'FLUTTER_ENGINE=' + ios_variant +
' | ' + pretty
'FLUTTER_ENGINE=' + ios_variant
]
RunCmd(command, cwd=ios_unit_test_dir, shell=True)

Expand Down