1351-1400 of 10000 results (25ms)
2012-10-12 §
02:01 <logmsgbot_> LocalisationUpdate failed: git pull of extensions failed [production]
01:21 <mutante> moving ancient copy of bugzilla out of /h/w/htdocs (to /root/backup on fenari temp. just in case) [production]
01:15 <mutante> deleting old document root dirs on fenari. /home/wikipedia/htdocs/noc and noc.old (use /h/w/common/docroot/noc and mediawiki-config repo) [production]
01:08 <mutante> restarting Apache on fenari for noc.wm change [production]
00:42 <Jeff_Green> switched payments back to pmtpa and increased TTL to 5 min [production]
00:37 <logmsgbot_> reedy synchronized php-1.21wmf1/extensions/Collection 'And back to master' [production]
00:31 <logmsgbot_> reedy synchronized php-1.21wmf1/extensions/Collection 'Revert due to error' [production]
00:29 <logmsgbot_> reedy synchronized php-1.21wmf1/extensions/Collection 'Update collection to master' [production]
00:09 <cmjohnson1> storage3 being reimaged [production]
2012-10-11 §
23:19 <mutante> manually copy index.html for noc from /common/docroot/noc to /h/w/htdocs/noc/ [production]
23:15 <logmsgbot_> dzahn synchronized docroot/noc/index.html [production]
23:13 <logmsgbot_> dzahn synchronized docroot/noc/index.html [production]
23:08 <logmsgbot_> olivneh synchronized php-1.21wmf1/extensions/E3Experiments [production]
22:16 <logmsgbot_> olivneh synchronized php-1.21wmf1/extensions/E3Experiments [production]
21:58 <logmsgbot_> bsitu Finished syncing Wikimedia installation... : Update ArticleFeedbackv5, MoodBar, PageTriage to master [production]
21:36 <logmsgbot_> bsitu Started syncing Wikimedia installation... : Update ArticleFeedbackv5, MoodBar, PageTriage to master [production]
20:48 <Jeff_Green> flipping payments.wikimedia.org to eqiad cluster to live-test [production]
20:12 <mutante> rebooting analytics1007 [production]
19:02 <notpeter> stopping puppet on brewster [production]
18:50 <RobH> i need to pull virt1001-1003 out of nagios later, as they are now renamed [production]
16:52 <notpeter> restarting pdns on ns0 [production]
16:51 <logmsgbot_> preilly synchronized php-1.21wmf1/extensions/MobileFrontend 'update for landing page' [production]
16:50 <notpeter> dist-upgrade and reboot on oxygen [production]
16:49 <K4-713> updated the payments cluster to 4d0223ebb48f9b [production]
16:39 <RobH> authdns update for pc1001-1003 mgmt ip [production]
15:35 <RobH> working on analytics1007 per rt 3582 [production]
14:56 <RobH> shutting down db1012 to remove evaluation ssd card [production]
14:14 <paravoid> apt: including php-parsekit, python-async, python-git, python-gitdb, python-smmap backports for precise [production]
08:59 <apergos> restarted swift proxy on ms-fe3 and ms-fe2 with some delay between. note I had to shoot stale processes from Sept on all these boxes as well [production]
08:50 <apergos> restarted ms-fe4 swift-proxy [production]
04:13 <mutante> activated W3 Total Cache performance plugin in Wordpress (blog.wikimedia.org) [production]
04:12 <mutante> marmontel (blog): fixed some file permissions in ./blog/wp-content for www-data the wordpress caching plugin needs to be re-activated [production]
02:27 <logmsgbot_> LocalisationUpdate completed (1.21wmf1) at Thu Oct 11 02:27:51 UTC 2012 [production]
01:55 <LeslieCarr> moved traffic back to normal with authdns-scenario [production]
01:48 <LeslieCarr> reenabling nagios notifications [production]
01:22 <LeslieCarr> upgrading csw2-esams member 1 [production]
01:22 <LeslieCarr> upgrading csw2-esams member 0 [production]
01:13 <binasher> disabled nagios notifications [production]
01:04 <LeslieCarr> upgrading csw2-esams member 2 [production]
01:02 <LeslieCarr> upgrading csw2-esams member 3 [production]
00:44 <binasher> restarted pdns on ns2 [production]
00:44 <LeslieCarr> restarting dns on ns2 [production]
00:33 <LeslieCarr> draining esams via authdns-scenarios [production]
00:11 <logmsgbot_> reedy synchronized php-1.21wmf1/includes/api/ApiUpload.php [production]
2012-10-10 §
23:02 <RobH> db1008.mgmt returned to service [production]
23:02 <RobH> msw-a2-eqiad has a bad set of ports, needs replacement per rt 3683 [production]
22:46 <RobH> reboot seems to have restored system (is posting) but drac still not responding, possible bad cable, will test shortly [production]
22:40 <RobH> db1008 being cold reset (removing power cords) [production]
22:05 <mutante> apt-get dist-upgrade on marmontel (blog) [production]
21:20 <LeslieCarr> undraining esams via authdns-scenario normal [production]