[13.07.2020] Added "t3version" header evaluation
I have noticed, that some TYPO3 website owners block the IP address of the t3versions server to prevent the major version from being shown public. I fully understand the motivation behind this action, but blocking IPs will only work temporary, since a full scan of all websites use several servers with different IP addresses.
If a website owner want to disable the exposure of the version number, this can now easily be configured in TYPO3 by settings an additional header as shown in the example below:
config.additionalHeaders.90.header = t3versions: hide-major
[12.07.2020] Re-Check of all TYPO3 websites in the database
While optimizing the TYPO3 detection engine I discovered 2 major problems which sadly resulted in distorted result in all previous statistics. The engine did not cover the following scenarios known TYPO3 websites in the database:
- Check discovered, that the website is offline / domain unreachable, but did not remove it from the database.
- Check followed domain redirects but did not remove known website, when target of redirect is not using TYPO3
Both scenarios are now covered and the rescan resulted in a decrease of 51635 TYPO3 websites in the database of which 28545 are offline/unreachable.
[07.04.2020] New statistics for Q1 2020
I have optimized the scan infrastructure, so more processes are automated. So not it is possible to spin up a various amount of scan engines based on Docker, that all connect to a central Redis crawling queue.
As a result of the optimization, I scanned multiple lists containing 49.511.476 european domains in total and also did a rescan of all TYPO3 websites in the database.
[04.12.2019] First Patch Level Statistics
All available TYPO3 websites in the database were analyzed for the TYPO3 Patch Level Version in use. For websites, where it is clearly possible to analyze the TYPO3 patch level version, only 49.211 (23.91%) out of 205.789 websites use a secure TYPO3 version.
[16.09.2019] Re-Check of all TYPO3 websites in the database
I updated the TYPO3 fingerprinting and also added an additional check, which tries to identify the new CMS a website uses when it stopped using TYPO3. It was no surprise to me, that about 33% of all websites that stopped using TYPO3 began to use WordPress. Also good to see, that a lot of websites updated either to TYPO3 8.7 (7437) or 9.5 (5270)
[03.05.2019] Checked list with 140+ mio domains and ~285000 known TYPO3 sites
Since the last Re-Check in January, I bought a big list with over 140 mio. domains and scanned all of them. This scan did find 10735 additional TYPO3 Websites.
Also MaxServ contributed a list of ~285000 known TYPO3 sites. This check resulted in 45.905 new sites in the database.
[05.01.2019] Re-Check of all TYPO3 websites in the database
I did a re-check of all TYPO3 websites in the database and added 2 new charts. 2284 new sites using TYPO3 were found since the last statistics as a result of manual domain checks of users using t3versions.
You may notice, that there are some really low versions in the "TYPO3 update target version" charts available. Since I also optimized the TYPO3 version identification process in t3versions, some sites with a previous "N/A" version are now identified with the actual TYPO3 version and therefore saved as e.g. "Updated N/A -> 3.7" in the database.
[10.07.2018] Public release and updated support chart
I released t3version to the public and updated the "Support vs. unsupported versions" chart to show TYPO3 6.2 as ELTS supported.
[07.07.2018] Remove duplicates and optimize TYPO3 version check
I found out, that 12.297 domains were double indexed, because those sites were indexed starting with and without "www.". I optimized the TYPO3 version check to respect this scenario in the future and removed all duplicates. The total amount of TYPO3 websites is now 292.629
[04.07.2018] Check of list with 48.146.633 finished
The check of all 48.146.633 domains is finished. As a result, 304.926 websites was found with TYPO3 as content management system.
[16.06.2018] Increased amount of worker processes and added additional crawling server
Finished crawling 1 mio. domains from the big domain list.
With the previous configuration of the queue system, about 8 domains could be checked per second on the production server (4 CPU cores, 12 GB RAM). Since the total CPU load of the system was really low, I increased the amount of worker processes of the queue system to 64, which increased the the crawling speed to about 23 domains per second.
I also added an additional crawling server (6 CPU cores, 24 GB RAM), which uses 80 worker processes. The server has access to the main database and can check about 34 domains per second.
[14.06.2018] Bought a list with 48.146.633 european domains
I bought list with 48.146.633 european domain names. Started crawling the domains for TYPO3 websites.
[31.05.2018] Bought a list with 481.883 domains using TYPO3
I bought a list with 481.883 domains using TYPO3 from an online service specialized on web technology analysis. As a result, the total amount of websites using TYPO3 increased 183.024. After analyzing the domain list i noticed, that it contained many domains, which just were used as redirects for a main domain (e.g. www.domain.dk -> www.domain.com).
[20.05.2018] Bought a list with 24.092 domains using TYPO3
I bought a list with 24.092 domains using TYPO3 from an online service specialized on web technology analysis. As a result, the total amount of websites using TYPO3 increased to 18.389
[20.05.2018] Implemented queue system for website crawling
Implemented a queue system, which handles the crawling queue and works with multiple CPU cores.
[15.05.2018] Imported a lists with 400 TYPO3 websites
I imported a list with about 400 TYPO3 websites. From the 400 websites, 95% were still using TYPO3
[14.05.2018] Added basic statistics
Added some basic statistics about TYPO3 version usage
[07.05.2018] Deployment to server
Added auto deployment and deployed first version to production server
[29.04.2018] Initial version
I started developing the first version of t3versions.