601-650 of 10000 results (20ms)
2013-03-05 §
02:49 <logmsgbot_> LocalisationUpdate completed (1.21wmf11) at Tue Mar 5 02:48:58 UTC 2013 [production]
02:28 <logmsgbot_> LocalisationUpdate completed (1.21wmf10) at Tue Mar 5 02:28:16 UTC 2013 [production]
02:03 <K4-713> updated payments cluster to 5361c795f4bf31980f [production]
00:46 <mutante> fixing exim/mail permissions on neon to make Icinga paging work. missing +x on /var/log/exim4 dir [production]
2013-03-04 §
23:12 <mutante> starting dhcp3 on brewster [production]
23:10 <LeslieCarr> p.s. bonding on foundry sucks [production]
23:10 <LeslieCarr> bringing down virt2 network to bond it [production]
22:18 <mutante> gracefulling eqiad Apaches to push indiawikipedia redirect [production]
22:17 <logmsgbot_> dzahn gracefulled all apaches [production]
22:03 <logmsgbot_> py synchronized wmf-config/db-pmtpa.php 'shuffling around pmtpa enwiki slaves' [production]
21:50 <LeslieCarr> messing around with the network of virt2 - expect possible pmtpa labs network issues [production]
21:50 <ottomata> staring puppet back up on oxygen, packet loss is better after disabling a filter [production]
21:30 <cmjohnson1> authdns update [production]
21:17 <logmsgbot_> reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki and mediawikiwiki to 1.21wmf11 [production]
21:16 <logmsgbot_> reedy synchronized php-1.21wmf11 [production]
21:15 <logmsgbot_> reedy synchronized php-1.21wmf10 [production]
21:14 <RobH> shutting down professor for a fan swap, should be a short window [production]
21:11 <cmjohnson1> authdns update [production]
21:05 <ottomata> stopping puppet on oxygen to see if we can disable filters and reduce packet loss [production]
20:55 <Ryan_Lane> restarted icinga [production]
20:52 <logmsgbot_> reedy Finished syncing Wikimedia installation... : Rebuild Message Cache for 1.21wmf11 [production]
20:52 <RobH> had to restart morebots, but it wasnt really dead, opps [production]
20:51 <RobH> had to restart icinga [production]
20:51 <RobH> Why arent you taking all the !log [production]
20:50 <RobH> i hate you morebots [production]
20:28 <logmsgbot_> reedy Started syncing Wikimedia installation... : Rebuild Message Cache for 1.21wmf11 [production]
19:55 <logmsgbot_> reedy Started syncing Wikimedia installation... : Build 1.21wmf11 message cache [production]
19:41 <logmsgbot_> reedy synchronized php-1.21wmf11/ [production]
19:34 <logmsgbot_> reedy synchronized wmf-config/ [production]
19:34 <logmsgbot_> reedy synchronized live-1.5/ [production]
19:33 <logmsgbot_> reedy synchronized docroot [production]
19:30 <hashar> on gallium: stopping puppet, cleaning defunct and restarting puppet [production]
19:23 <logmsgbot_> reedy synchronized wmf-config/ [production]
19:20 <mutante> installing package upgrades on sockpuppet [production]
19:13 <sbernardin> taking down amaranth for hw replacement: https://rt.wikimedia.org/Ticket/Display.html?id=4497 [production]
18:44 <cmjohnson1> authdns update [production]
18:17 <K4-713> updated payments cluster to 059969c3d29f685c [production]
17:44 <cmjohnson1> authdns update [production]
16:56 <logmsgbot_> py synchronized wmf-config/db-eqiad.php 'depooling db1009 while investigating error' [production]
16:24 <sbernardin> rebooting db70 to reset idrac: https://rt.wikimedia.org/Ticket/Display.html?id=4628 [production]
16:14 <sbernardin> db27 being taken down for decom per: https://rt.wikimedia.org/Ticket/Display.html?id=4623 [production]
10:12 <apergos> cleared some space on brewster by rmeoving old squid access/storage logs (2.2gb worth in 3 files!), its root partition was full, restarted squid [production]
10:12 <apergos> reinstalled wikimedia-app-taskserver on fenari (how did it get removed?), needed for various cron jobs among other things [production]
02:04 <logmsgbot_> LocalisationUpdate failed (1.21wmf10) at Mon Mar 4 02:04:10 UTC 2013 [production]
2013-03-03 §
02:25 <LocalisationUpdate> completed (1.21wmf10) at Sun Mar 3 02:25:44 UTC 2013 [production]
2013-03-02 §
02:27 <LocalisationUpdate> completed (1.21wmf10) at Sat Mar 2 02:27:35 UTC 2013 [production]
01:17 <mutante> powercycling emery [production]
00:35 <mutante> rebooting emery for kernel upgrade [production]
00:29 <mutante> installing package upgrades on emery [production]
2013-03-01 §
23:59 <cmjohnson1> authdns update [production]