Thanks to a benevolent user, timops96,
I found and fixed an issue in WordPress’ registration function.
It was using the mail() function of PHP which any decent webserver(+PHP) will deactivate it.
I replaced it with smtpmail and did some additional tricks and now it seems to work.
Have fun registering !
Here is a link to registration:
Register