- the tests themselves are not quite the right ones to reveal true performance (ie the servers are running better than the tests suggest)
- these tests were done on a system with separate db & web servers
- refine the tests used
- set up a simple mechanism to share results and server configurations in a shared database
Hugh
Results of All Benchmarking Tests till date on load testing
Scenario | Time Run | Script Execution Time |
Memory Total MB |
Memory Growth MB |
Files Included |
DB Queries |
Log Writes |
Cache Hits |
Cache Misses |
Ticks | User | Sys | cuser | csys | Load Average |
Function Calls k |
Regex k |
Read from Disk k |
Write to Disk k |
get record calls k |
insert record calls k |
update record calls k |
Maximum concurrent users |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
during load test with internal cache | 30/09 22:45 | 27.9 | 1.18 | 1.12 | 35 | 0 | 0 | 45130 | 3 | 2788 | 931 | 261 | 0 | 0 | 2.38 | 503 | 7.9 | 20.65 | 6.7 | 22.57 | 0.48 | 0.12 | 127 |
during load test with memcached - run 2 | 30/09 22:44 | 28.4 | 1.18 | 1.12 | 36 | 0 | 0 | 4953 | 0 | 2837 | 876 | 313 | 0 | 0 | 1.85 | 538.5 | 7.35 | 24.05 | 7.05 | 2.48 | 0.47 | 0.11 | 73 |
during load test with memcached | 30/09 22:42 | 28.4 | 1.18 | 1.12 | 36 | 0 | 0 | 5673 | 0 | 2843 | 906 | 302 | 0 | 0 | 2.39 | 582 | 7.3 | 27.05 | 6.45 | 2.84 | 0.55 | 0.12 | 79 |
during load test with memcached disabled | 30/09 22:35 | 27.9 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2795 | 827 | 304 | 0 | 0 | 2.87 | 550 | 8.25 | 14.9 | 7.25 | 2.3 | 0.82 | 0.17 | 76 |
memcached disabled - run 2 | 30/09 22:19 | 27.9 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2786 | 848 | 301 | 0 | 0 | 0.38 | 858.5 | 13.75 | 30.75 | 9.4 | 2.28 | 0.8 | 0.17 | 107 |
memcached disabled - run 1 | 30/09 22:18 | 27.6 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2765 | 854 | 304 | 0 | 0 | 0.35 | 836.5 | 13.8 | 30.9 | 9.45 | 2.27 | 0.8 | 0.17 | 107 |
max clients set to 100 - run 2 | 30/09 22:16 | 28.2 | 1.18 | 1.12 | 36 | 0 | 0 | 7783 | 0 | 2817 | 874 | 333 | 0 | 0 | 0.27 | 822 | 13.9 | 33.05 | 9.45 | 3.89 | 0.81 | 0.16 | 113 |
max clients set to 100 - run 1 | 30/09 22:16 | 28.4 | 9.61 | 9.31 | 36 | 0 | 0 | 7423 | 0 | 2838 | 909 | 323 | 0 | 0 | 0.15 | 797.5 | 14.75 | 32.45 | 9.45 | 3.71 | 0.8 | 0.16 | 113 |
max clients set to 175 - run 1 | 30/09 22:14 | 28.4 | 9.61 | 9.31 | 36 | 0 | 0 | 9183 | 0 | 2843 | 929 | 318 | 0 | 0 | 0.2 | 781.5 | 13.05 | 31.75 | 9.35 | 4.59 | 0.84 | 0.17 | 113 |
memcached with cache set to 512MB - run 2 | 30/09 22:03 | 27.6 | 1.18 | 1.12 | 36 | 0 | 0 | 9273 | 0 | 2763 | 906 | 309 | 0 | 0 | 0.22 | 826.5 | 14.8 | 30.5 | 9.25 | 4.64 | 0.84 | 0.16 | 115 |
memcached with cache set to 512MB | 30/09 22:03 | 27.6 | 1.4 | 1.34 | 36 | 0 | 0 | 8963 | 0 | 2759 | 882 | 320 | 0 | 0 | 0.18 | 827.5 | 13 | 31.7 | 9.4 | 4.48 | 0.85 | 0.16 | 114 |
memcached test - run 3 | 30/09 21:52 | 27.7 | 1.18 | 1.12 | 36 | 0 | 0 | 7903 | 0 | 2772 | 878 | 330 | 0 | 0 | 0.33 | 817.5 | 14.65 | 31.3 | 9.4 | 3.95 | 0.83 | 0.16 | 113 |
memcached test - run 2 | 30/09 21:51 | 28.3 | 1.54 | 1.42 | 36 | 0 | 0 | 8900 | 3 | 2827 | 851 | 366 | 0 | 0 | 0.27 | 790 | 14.45 | 17.75 | 9.35 | 4.45 | 0.86 | 0.17 | 102 |
internal moodle cache enabled - run 2 | 30/09 21:42 | 27.7 | 1.18 | 1.12 | 35 | 0 | 0 | 64890 | 3 | 2769 | 945 | 276 | 0 | 0 | 0.2 | 806 | 14.4 | 29.85 | 9.15 | 32.45 | 0.82 | 0.17 | 192 |
internal moodle cache enabled | 30/09 21:41 | 28.3 | 1.18 | 1.12 | 35 | 0 | 0 | 66390 | 3 | 2829 | 949 | 287 | 0 | 0 | 0.16 | 798.5 | 14.7 | 30.45 | 9.2 | 33.2 | 0.86 | 0.17 | 196 |
maximum number of connections set to 200 Second Run | 30/09 21:38 | 28.2 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2818 | 857 | 293 | 0 | 0 | 0.39 | 813.5 | 13.7 | 30.55 | 9.4 | 2.28 | 0.82 | 0.17 | 106 |
maximum number of connections set to 200 First Run | 30/09 21:37 | 27.6 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2762 | 874 | 295 | 0 | 0 | 0.25 | 813.5 | 14.5 | 31.9 | 9.35 | 2.24 | 0.89 | 0.16 | 110 |
key_buffer_size = 32M my.conf Second Run | 30/09 21:33 | 27.8 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2783 | 874 | 288 | 0 | 0 | 0.24 | 827.5 | 14.25 | 32.2 | 9.35 | 2.27 | 0.81 | 0.17 | 108 |
key_buffer_size = 32M my.conf First Run | 30/09 21:32 | 27.4 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2737 | 867 | 297 | 0 | 0 | 0.2 | 826 | 14.35 | 32 | 9.65 | 2.31 | 0.89 | 0.17 | 111 |
table_cache = 512 my.conf Second Run | 30/09 21:27 | 27.8 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2783 | 860 | 292 | 0 | 0 | 0.19 | 833 | 14.45 | 32.95 | 9 | 2.32 | 0.81 | 0.17 | 109 |
table_cache = 512 my.conf Frist Run | 30/09 21:26 | 28 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2798 | 850 | 310 | 0 | 0 | 0.29 | 837.5 | 13.75 | 30.85 | 9.3 | 2.33 | 0.88 | 0.17 | 109 |
query_cache_min_res_unit= 2k my.conf Second Run | 30/09 21:24 | 27.3 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2732 | 847 | 301 | 0 | 0 | 0.32 | 814 | 14.4 | 32 | 9.5 | 2.28 | 0.84 | 0.17 | 109 |
query_cache_min_res_unit= 2k my.conf First Run | 30/09 21:23 | 27.7 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2768 | 878 | 291 | 0 | 0 | 0.2 | 804 | 14.45 | 32.15 | 9.55 | 2.32 | 0.91 | 0.18 | 112 |
query_cache_type = 1 my.conf Second Run | 30/09 21:13 | 27.6 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2759 | 844 | 297 | 0 | 0 | 0.21 | 821.5 | 14.35 | 32.85 | 9.45 | 2.32 | 0.83 | 0.17 | 110 |
query_cache_type = 1 my.conf First Run | 30/09 21:12 | 28.3 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2827 | 853 | 319 | 0 | 0 | 0.2 | 769 | 14.5 | 22.65 | 8.25 | 2.34 | 0.89 | 0.18 | 100 |
query_cache_size = 36M Second Run | 30/09 21:09 | 27.7 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2773 | 846 | 305 | 0 | 0 | 0.18 | 840 | 13.7 | 30.6 | 8.9 | 2.28 | 0.82 | 0.17 | 106 |
query_cache_size = 36M First Run | 30/09 21:08 | 27.6 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2756 | 869 | 308 | 0 | 0 | 0.33 | 845 | 14.8 | 30.95 | 9.1 | 2.27 | 0.89 | 0.18 | 110 |
memory_limit set o 40MB in php.ini Second Run | 30/09 21:03 | 27.9 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2790 | 874 | 292 | 0 | 0 | 0.38 | 759.5 | 14.4 | 32.35 | 9.35 | 2.3 | 0.83 | 0.18 | 108 |
memory_limit set o 40MB in php.ini | 30/09 21:01 | 27.7 | 1.39 | 1.32 | 35 | 0 | 0 | 0 | 0 | 2775 | 877 | 291 | 0 | 0 | 0.29 | 808.5 | 14.2 | 32.55 | 9.25 | 2.32 | 0.87 | 0.18 | 110 |
IfModule mod_expires.c | 30/09 00:45 | 28.1 | 1.39 | 1.32 | 35 | 0 | 0 | 0 | 0 | 2807 | 855 | 305 | 0 | 0 | 0.12 | 866 | 14.5 | 31.45 | 9.4 | 2.28 | 0.79 | 0.18 | 108 |
register_globals=off | 30/09 00:38 | 27.6 | 1.39 | 1.32 | 35 | 0 | 0 | 0 | 0 | 2755 | 862 | 310 | 0 | 0 | 0.18 | 845 | 14.7 | 31.05 | 9.05 | 2.28 | 0.87 | 0.18 | 109 |
with apc installed Second Run | 30/09 00:35 | 27.6 | 1.17 | 1.11 | 35 | 0 | 0 | 0 | 0 | 2765 | 866 | 299 | 0 | 0 | 0.1 | 796 | 14.7 | 31.3 | 9.3 | 2.28 | 0.8 | 0.18 | 107 |
with apc installed | 30/09 00:34 | 28 | 1.39 | 1.32 | 35 | 0 | 0 | 0 | 0 | 2803 | 867 | 304 | 0 | 0 | 0.11 | 835.5 | 14.7 | 31.15 | 9.25 | 2.3 | 0.87 | 0.18 | 110 |
initial test | 20/09 02:25 | 27.9 | 9.56 | 9.27 | 35 | 0 | 0 | 0 | 0 | 2795 | 845 | 324 | 0 | 0 | 0.13 | 814 | 13 | 21.15 | 8.1 | 1.87 | 0.75 | 0.18 | 92 |
second time | 16/09 03:08 | 27.9 | 14.69 | 14.12 | 35 | 0 | 0 | 0 | 0 | 2788 | 852 | 327 | 0 | 0 | 0.3 | 887.5 | 12.2 | 28.05 | 9.3 | 1.98 | 0.72 | 0.18 | 99 |
this is the initial run | 16/09 03:06 | 27.6 | 14.91 | 14.34 | 35 | 0 | 0 | 0 | 0 | 2761 | 853 | 335 | 0 | 0 | 0.19 | 859.5 | 12.1 | 20 | 8.35 | 2.06 | 0.68 | 0.18 | 89 |