Add code testing via phpunit and/or cypress

[parapente]

This is an effort to add automated testing of the code to ensure that the extension works whenever new patches are added to the codebase or the joomla framework changes.

There are few phpunit tests (ported from older tests that were present in the codebase) but almost all fail for now. It will need quite some effort to refactor the extension code to be able to test it effectivly using phpunit.

Cypress on the other hand can help us test the extension as it is running on an joomla instance like we would manually. I started adding some basic tests that are run in batches. I have created folders for each part of the package in /cypress/e2e that eventually will be populated with the tests.

I will add a checklist for everything that needs to be done and of course the last step would be to add github actions to run the tests whenever we push code to master.

[JLTRY]

Hello it seems to be a big work.
I wanted to try to use the tests.
Are there ready yet ?
I see that the file FOR_DEVELOPERS.md was modified for PHPUnit
Is it usable ?
I see also docker files for joomla cypress novnc , what are the files used for ?

[parapente]

For phpunit:

• You need to run composer install (or composer update if you get a warning about the lock file)
• composer test (to run the tests)

For cypress:

• Requirement: You need to have docker compose available on your system (with docker desktop on windows and mac, with docker desktop or docker repository in linux) Install instructions
• Copy the .env.4.4.example to .env (This will add joomla 4.4 with all the necessary requirements for the tests. The other 2 example files are for joomla 5.4 and 6 respectively. I am still testing with 4.4)
• docker compose --profile testing up -d
• If all went well you now can open http://localhost:8090 and connect to novnc which will have cypress desktop opened. You select end 2 end tests and the browser of your choice (I have only tested chrome for now) and then select a spec file to run the tests. It only contains a small amount of tests but I will eventually add a lot more.
• Using http://localhost:8080 you can access the joomla site and if you run docker compose up -d phpmyadmin you will have on http://localhost:8888 phpmyadmin available to access the database.

[JLTRY]

@parapente for phpunit I have an error at st step 1)

Tests\Unit\Helper\AttachmentsInstallTest::testInstallAttachmentsPermissionsMethodExists
PHPUnit\Framework\MockObject\Generator\ReflectionException: Class "Joomla\CMS\User\User" does not exist

[JLTRY]

I fixed last error by adding link from temp\joomla\libraries\src to joomla 4.x library src path

[JLTRY]

Next error is

Tests\Unit\Helper\AttachmentsPermissionsTest::testGetActionsSuperAdmin
Error: Call to a member function load() on false

F:\Sites\site OVH JLT local\joomla_4.0\libraries\src\User\User.php:819`

[parapente]

@JLTRY yes that is the error I'm getting too. The summary gives:

Tests: 222, Assertions: 265, Errors: 75, Deprecations: 16

but most of those that pass as quite trivial tests. Another thing that will be of importance if we continue to work with phpunit is that the joomla framework packages have different numbering from the joomla cms releases:

Joomla Framework	Joomla CMS
2.x	4.x
3.x	5.x
4.x	6.x

So for the packages that I have added as requirements in composer.json we should use the joomla 6 code in /temp/joomla.

[JLTRY]

I have made a link to joomla_6 libraries/src. It still fails
I commented the lines in bootstrap

/*if (class_exists('\\Joomla\\CMS\\Factory')) {
   // If running within a Joomla installation, use the Joomla factory
   // This is for when tests are run from within a Joomla instance
} else {
   // Setup Joomla framework for standalone testing
   // This is where we'd initialize mock services if needed
}*/

I have got this error
Error: mysqli object is not fully initialized
Let me know if you go further

[JLTRY]

* `docker compose --profile testing up -d`

* If all went well you now can open http://localhost:8090 and connect to novnc which will have cypress desktop opened. You select end 2 end tests and the browser of your choice (I have only tested chrome for now) and then select a spec file to run the tests. It only contains a small amount of tests but I will eventually add a lot more.

* Using http://localhost:8080 you can access the joomla site 

and if you run docker compose up -d phpmyadmin you will have on http://localhost:8888 phpmyadmin available to access the database.

I went well to step docker compose --profile testing up -d
For novnc I have a blank screen

For joomla I have the login prompt what is the pwd ?

[parapente]

@JLTRY If you get and empty desktop in novnc there is probably an error during cypress desktop startup. You can see the logs if you run docker compose logs cypress. You can retry to run cypress via docker compose restart cypress or docker compose up -d cypress.

The default usernames and passwords are the ones that appear in docker-compose.yml . For joomla you should use username: joomla and password: joomla_password (it had to be 12 characters or longer unfortunately).

[JLTRY]

@JLTRY If you get and empty desktop in novnc there is probably an error during cypress desktop startup. You can see the logs if you run docker compose logs cypress. You can retry to run cypress via docker compose restart cypress or docker compose up -d cypress.

Do you run cypress under windows /docker or under linux / docker ?

[parapente]

I am using linux / docker but using docker should give you the same result both on linux and windows. On windows is a little more complicated to get started with docker and a little less performant but it should be the same.

There is another way to run the cypress tests but it will require you to do some manual configuration. Instead of running docker compose --profile testing up -d you can omit the profile option and just give docker compose up -d. This will start only the apache/mariadb servers and you will have the website at http://localhost:8080. Then you can install the cypress desktop app in windows and configure it to find the application and the joomla url. The configuration file for the project is the file cypress.config.js in the root of the project. Please don't push changes to this file on the repository.

Cypress has two modes of running: headless (which just runs the tests and doesn't open an application window) and desktop. For now I am using the desktop mode to write the tests and when they are ready, using the same config we could add a github action to run the tests automatically on every push we make to the repository.