901-950 of 10000 results (14ms)
2012-05-01 §
17:51 <Ryan_Lane> make that virt0 [production]
17:51 <Ryan_Lane> switching the session cache back to filesystem on virt1, since it isn't working properly with memcache [production]
17:29 <maplebed> kicking nagios to check a change to fix the mobile LVS alert [production]
17:25 <logmsgbot_> nikerabbit synchronized php-1.20wmf2/extensions/TranslationNotifications/ 'Deploying TranslationNotifications code' [production]
17:08 <logmsgbot_> reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki to 1.20wmf2 [production]
16:27 <notpeter> starting innobackupex from db1034 to db53 for new s2 slave [production]
16:27 <notpeter> starting innobackupex from db57 to db52 for new s2 slave [production]
16:03 <notpeter> rebuilding db52 and db53 as s2 slaves [production]
15:47 <logmsgbot_> asher synchronized wmf-config/db.php 's1: raising db59,60 weights, pulling db53/54 for reuse' [production]
09:23 <logmsgbot_> reedy synchronized wmf-config/CommonSettings.php 'hewiki account creation high throttle limits' [production]
04:04 <Tim> on all apaches, running "chmod -R a+rX /usr/local/apache/common-local/" to clean up after killed rsyncs which left files unreadable [production]
02:23 <logmsgbot_> LocalisationUpdate completed (1.20wmf2) at Tue May 1 02:23:29 UTC 2012 [production]
02:21 <logmsgbot_> preilly synchronized php-1.20wmf1/extensions/MobileFrontend/specials/SpecialMobileFeedback.php [production]
02:14 <logmsgbot_> LocalisationUpdate completed (1.20wmf1) at Tue May 1 02:14:06 UTC 2012 [production]
02:06 <logmsgbot_> preilly synchronized php-1.20wmf1/extensions/MobileFrontend/specials/SpecialMobileOptions.php [production]
01:51 <Ryan_Lane> bringing up all labs instances with a 60 second lag [production]
01:40 <Ryan_Lane> rebooting virt0 [production]
01:35 <Ryan_Lane> rebooting virt3 [production]
01:33 <logmsgbot_> preilly synchronized php-1.20wmf1/extensions/MobileFrontend/HtmlFormatter.php [production]
01:26 <Ryan_Lane> rebooting virt5 [production]
01:18 <Ryan_Lane> rebooting virt4 [production]
01:03 <Ryan_Lane> rebooting virt2 [production]
00:51 <LeslieCarr> restarted swift-container-auditor on ms-be5 [production]
00:38 <logmsgbot_> tstarling synchronizing Wikimedia installation... : [production]
00:26 <Tim> removed large syslogs from mw60 and ran sync-common [production]
00:18 <Tim> on mw60 there was an actual directory at /usr/local/apache/common/php where a symlink should have been. fixed [production]
2012-04-30 §
23:58 <logmsgbot_> aaron synchronized php [production]
23:44 <RoanKattouw> Started Apache back up on mw60 [production]
23:39 <RoanKattouw> Running scap-1 on the Apaches with dsh [production]
23:38 <RoanKattouw> Moved /home/catrope/php-1.19 to /home/wikipedia/lazy-backups/php-1.19 [production]
23:38 <Reedy> mediawiki.org to 1.20wmf2 [production]
23:37 <logmsgbot_> reedy rebuilt wikiversions.cdb and synchronized wikiversions files: mw.org to 1.20wmf2 [production]
23:35 <RoanKattouw> Strike that, instead moving /home/w/common/php-1.19 to /home/catrope/php-1.19 [production]
23:34 <RoanKattouw> Removing /home/w/common/php-1.19 , NFS might freak out a bit [production]
23:31 <RoanKattouw> Removed php-1.19 from mw60 , synced it, and restarted Apache [production]
23:28 <RoanKattouw> Synced docroot and purged varnish for static-1.20wmf2, bits seems to be working for 1.20wmf2 now [production]
23:27 <RoanKattouw> mw60 has full disk, stopping Apache for now [production]
22:50 <Ryan_Lane> rebooting virt5 [production]
22:42 <Ryan_Lane> rebooting virt3 [production]
22:35 <Ryan_Lane> rebooting virt4 [production]
22:28 <Ryan_Lane> rebooting virt1 [production]
22:23 <Ryan_Lane> bringing down all instances (yay gluster) [production]
21:12 <pgehres> re-enabled Jenkins jobs on Aluminium after db1008 reboot [production]
21:11 <pgehres> CiviCRM back to normal after db1008 reboot [production]
21:07 <Jeff_Green> db1008 gets kernel update and reboot [production]
21:00 <pgehres> put CiviCRM on Aluminium in maintenance mode for db1008 reboot [production]
20:59 <logmsgbot_> reedy synchronized php-1.20wmf2/resources/startup.js 'touch' [production]
20:57 <pgehres> disabled all Jenkins jobs on Aluminium in prep for db1008 reboot [production]
20:50 <Jeff_Green> db1025 and storage3 get new kernels and reboot [production]
20:43 <notpeter> removing ssl4 from pool, stopping puppet on ssl4, adding 3rd udp2log host for testing, restarting nginx [production]