|
2011-10-28
§
|
| 14:27 |
<mutante> |
srv167 - Nagios reports HTTP down, apache2 running but not on 80, instead on high-port, part of testing? |
[production] |
| 14:13 |
<mutante> |
srv158 - host up, Apache down - running: bash -c /etc/init.d/apache2 stop; while ps -C apache2; do sleep 1; done; /etc/init.d/apache2 start |
[production] |
| 13:01 |
<RoanKattouw> |
Manually killing dormant Apache workers with 20%+ mem usage on srv188 |
[production] |
| 12:59 |
<RoanKattouw> |
And starting Apache again on srv189 |
[production] |
| 12:56 |
<RoanKattouw> |
Stopping and restarting Apache on srv189 to get rid of memory-hogging Apache processes |
[production] |
| 12:53 |
<RoanKattouw> |
Gracefulling Apache on srv189 |
[production] |
| 12:21 |
<catrope> |
synchronizing Wikimedia installation... : [[rev:101126|r101126]] |
[production] |
| 05:32 |
<Andrew> |
srv256: @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ |
[production] |
| 05:32 |
<andrew> |
synchronized wmf-config/CommonSettings.php 'Updated $wgMoodBarConfig[infoUrl]' |
[production] |
| 03:48 |
<Jeff_Green> |
restarted crashed mysql on payments1, repaired replication |
[production] |
| 02:20 |
<LocalisationUpdate> |
completed (1.18) at Fri Oct 28 02:22:41 UTC 2011 |
[production] |
| 00:53 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.php 'weekly update to mobile frontend' |
[production] |
| 00:19 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.php 'weekly update to mobile frontend' |
[production] |
|
2011-10-27
§
|
| 23:45 |
<Ryan_Lane> |
setting shell to sillyshell for fvassard and pdhanda, since they no longer need shell access to formey |
[production] |
| 23:41 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.php 'weekly update to mobile frontend' |
[production] |
| 23:33 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.php 'weekly update to mobile frontend' |
[production] |
| 23:20 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.php 'weekly update to mobile frontend' |
[production] |
| 23:16 |
<reedy> |
synchronized php-1.18/extensions/CodeReview/backend/CodeComment.php '[[rev:10185|r10185]]' |
[production] |
| 23:16 |
<reedy> |
synchronized php-1.18/extensions/CodeReview/backend/CodeRevision.php '[[rev:10185|r10185]]' |
[production] |
| 22:04 |
<LeslieCarr> |
the ganglia link on noc.wikimedia.org points to ganglia3 instead of ganglia |
[production] |
| 21:57 |
<RoanKattouw> |
Fixing LU perms again, hopefully this is the last time with the cron job user fix now in puppet |
[production] |
| 21:41 |
<reedy> |
synchronizing Wikimedia installation... : Pushing [[rev:101067|r101067]] numerous zh fixes |
[production] |
| 21:32 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/views/layout/_footmenu_default.html.php 'weekly update to mobile frontend' |
[production] |
| 21:24 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/stylesheets/common.css 'weekly update to mobile frontend' |
[production] |
| 21:00 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.php 'weekly update to mobile frontend' |
[production] |
| 20:57 |
<catrope> |
synchronizing Wikimedia installation... : Update MoodBar to trunk state, deploying feedback dashboard |
[production] |
| 20:51 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.php 'weekly update to mobile frontend' |
[production] |
| 20:47 |
<RoanKattouw> |
Applying schema changes for MoodBar on enwiki |
[production] |
| 20:36 |
<RoanKattouw> |
Applying schema changes for MoodBar on testwiki |
[production] |
| 18:30 |
<notpeter> |
ns2 dead and down. rebooting |
[production] |
| 14:53 |
<RoanKattouw> |
Restarting lsearch on search1 and search3 because rainman-sr told me to |
[production] |
| 10:19 |
<mutante> |
Bugzilla back upgraded with kernel and mysql, looks fine |
[production] |
| 10:08 |
<mutante> |
kaulen (Bugzilla) - dist-upgrade/kernel,reboot (scheduled maintenance) |
[production] |
| 03:16 |
<LocalisationUpdate> |
completed (1.18) at Thu Oct 27 03:19:13 UTC 2011 |
[production] |
| 03:13 |
<Tim> |
running iperf tests on eqiad internal network to get baseline performance |
[production] |
| 03:07 |
<Tim> |
taking mw1071 and mw1072 for udp2log testing |
[production] |
| 02:45 |
<LocalisationUpdate> |
completed (1.18) at Thu Oct 27 02:47:47 UTC 2011 |
[production] |
| 02:20 |
<LocalisationUpdate> |
completed (1.18) at Thu Oct 27 02:22:56 UTC 2011 |
[production] |
| 00:34 |
<neilk> |
synchronizing Wikimedia installation... : MFT @ [[rev:99799|r99799]], to ensure that UploadWizard multi-file select is off by default. Was accidentally on by default in last push, and it still needs work. |
[production] |
| 00:15 |
<reedy> |
synchronized php-1.18/extensions/CodeReview/ui/CodeRevisionTagView.php '[[rev:100913|r100913]]' |
[production] |
|
2011-10-26
§
|
| 23:55 |
<neilk> |
synchronizing Wikimedia installation... : deploying UploadWizard from [[rev:99547|r99547]] trunk (10 Oct 2011) to fix some regressions |
[production] |
| 23:07 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/views/layout/application.html.php 'weekly update to mobile frontend' |
[production] |
| 23:06 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/javascripts/opensearch.js 'weekly update to mobile frontend' |
[production] |
| 22:46 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/views/layout/application.html.php 'weekly update to mobile frontend' |
[production] |
| 22:40 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/stylesheets/common.css 'weekly update to mobile frontend' |
[production] |
| 22:21 |
<preilly> |
synchronized php-1.18/extensions/MobileFrontend/MobileFrontend.i18n.php 'weekly update to mobile frontend' |
[production] |
| 21:47 |
<notpeter> |
restarting spence to be able to actually run puppet and to slaughter zombies |
[production] |
| 21:12 |
<notpeter> |
restarting nagios |
[production] |
| 20:11 |
<preilly> |
synchronized wmf-config/InitialiseSettings.php 'Remove beta logo' |
[production] |
| 19:32 |
<preilly> |
finish pushing weekly mobile frontend changes |
[production] |