[testing] Improve running browser tests locally. #25013
Conversation
This changes how the browser test runner works by default when only a browser executable is passed into EMTEST_BROWSER (e.g. EMTEST_BROWSER=chrome). The test runner will now automatically: - Use the same options used in CircleCI (for chrome and firefox) - Create and delete a temporary profile directory - Terminate the browser These changes make it much easier to reproduce the circle ci environment locally and make running a test multiple times more reliable. To make it easier to also run in "headed" mode, I added a `--headed` option. Also, if the user specifies EMTEST_BROWSER with options the old behavior will be used and a custom start command can be used. Last, two flags were removed from chrome: - `--disk-cache-dir` this was causing errors (on CI and locally) - `--incognito` not allowed in corporate environments and not needed since the profile directory is automatically deleted.
| @@ -2335,6 +2370,15 @@ def browser_restart(cls): | |||
| logger.info('Browser did not respond to `terminate`. Using `kill`') | |||
| cls.browser_proc.kill() | |||
| cls.browser_proc.wait() | |||
| if cls.browser_data_dir: | |||
| utils.delete_dir(cls.browser_data_dir) | |||
| cls.browser_data_dir = None | |||
There was a problem hiding this comment.
It might be useful for debugging to leave this around after terminate, and instead clean it up on startup.
- add option to enable/disable auto browser config - switch to --headless flag - fix checking common.EMTEST_BROWSER since it's updated later
|
I still need to fix the profile dir and I also totally missed that for firefox we create a profile file with some settings. I need to add that in here... |
Maybe we can find equivalent command line flags? |
-copy firefox user.js file to profile
There weren't equivalents a few years ago, and from a quick search I don't see anything has changed. |
|
To summarize changes:
|
This changes how the browser test runner works by default. It will now automatically detect the browser type and:
Adds two new options:
EMTEST_BROWSER_AUTO_CONFIGor--browser-auto-config: disables the automatic configuration,EMTEST_HEADLESSor--headless: run the test in headless modeLast, two flags were removed from chrome:
--disk-cache-dirthis was causing errors (on CI and locally)--incognitonot allowed in corporate environments and not needed since the profile directory is automatically deleted.These changes make it much easier to reproduce the circle ci environment locally and make running a test multiple times more reliable.