51-100 of 10000 results (14ms)
2014-01-21 §
02:29 <ori> restarting gitblit using new upstart job def (I3f32dedf1) [production]
02:28 <LocalisationUpdate> completed (1.23wmf11) at 2014-01-21 02:28:40+00:00 [production]
02:15 <LocalisationUpdate> completed (1.23wmf10) at 2014-01-21 02:15:20+00:00 [production]
2014-01-20 §
13:51 <akosiaris> finished virt1000 reinitalization from virt0 [production]
13:51 <akosiaris> started virt1000 reinitialization from virt0 [production]
13:44 <akosiaris> stopped, backed up virt1000 and virt0 and started them up. Preparing for reinitializing virt1000 from virt0 [production]
11:13 <akosiaris> ms-be1003 kernel log show first lockups around Jan 19 06:30 UTC being XFS related. [production]
11:00 <ori> restarting Gitblit on antimony to test upstart script [production]
10:58 <akosiaris> powercycling ms-be1003. Console full of messages BUG: soft lockup - CPU#stuck for #s [production]
02:40 <LocalisationUpdate> ResourceLoader cache refresh completed at 2014-01-20 02:40:25+00:00 [production]
02:22 <LocalisationUpdate> completed (1.23wmf11) at 2014-01-20 02:22:28+00:00 [production]
02:12 <LocalisationUpdate> completed (1.23wmf10) at 2014-01-20 02:12:45+00:00 [production]
02:12 <springle> synchronized wmf-config/db-eqiad.php 'repool db1022, db1039 full steam' [production]
01:40 <springle> synchronized wmf-config/db-eqiad.php 'repool db1039, warm up' [production]
2014-01-19 §
18:59 <faidon> synchronized wmf-config/InitialiseSettings.php 'switch wmgRC2UDPAddress back to ekrem' [production]
18:56 <faidon> updated /a/common to {{Gerrit|I48a026eb8}}: Revert wmgRC2UDPAddress to ekrem [production]
18:48 <paravoid> cr1-sdtpa/cr2-pmtpa: rolling back config to pre-Tampa outage [production]
11:01 <ori> graceful'd apache on antimony to verify Ibd528f8f2 [production]
07:11 <springle> synchronized wmf-config/db-eqiad.php 'depool db1022' [production]
07:04 <springle> xtrabackup clone db1007 to db1039 [production]
06:28 <springle> synchronized wmf-config/db-pmtpa.php 's7 master update' [production]
05:01 <springle> synchronized wmf-config/db-eqiad.php 'promote db1033 s7 master' [production]
04:09 <ebernhardson> cherry-picked security fix to Flow for php-1.23wmf11 [production]
03:58 <ebernhardson> synchronized php-1.23wmf11/extensions/Flow [production]
03:19 <springle> synchronized wmf-config/db-eqiad.php 'depool db1024' [production]
03:13 <springle> synchronized wmf-config/db-eqiad.php 'repool db1028' [production]
03:12 <LocalisationUpdate> ResourceLoader cache refresh completed at 2014-01-19 03:12:20+00:00 [production]
02:58 <p858snake|l> [09:59AEST] <bd808> !log Updated kibana to bef3db2 [production]
02:56 <scfc_de> Restarted morebots (production-logbot) [production]
2014-01-18 §
15:38 <springle> xtrabackup clone db1007 to db1028 [production]
15:31 <springle> synchronized wmf-config/db-eqiad.php 'depool db1028' [production]
15:14 <springle> synchronized wmf-config/db-eqiad.php 'warm up db1033 in s7' [production]
11:04 <springle> xtrabackup clone db1007 to db1033 [production]
10:52 <springle> synchronized wmf-config/db-eqiad.php 'repool db1041' [production]
10:35 <paravoid> powercycling ms-be1012; down for 12h, console unresponsive [production]
03:52 <Coren> marc (on neon) [production]
03:50 <Coren> marc truncated /var/log/ganglia/ganglia_parser.log again. Check incoming email to ops-l [production]
03:39 <ori> synchronized php-1.23wmf11/extensions/UniversalLanguageSelector 'Update ULS to master for I862b01e6b (@embed fix)' [production]
03:38 <ori> synchronized php-1.23wmf11/extensions/UniversalLanguageSelector/resources/js/ext.uls.webfonts.js 'Update UniversalLanguageSelector to master for I2da436caa: Wait till rendering thread completion before applying webfonts (Bug: 59958)' [production]
03:32 <ori> synchronized php-1.23wmf10/extensions/UniversalLanguageSelector 'Update ULS to master for I862b01e6b (@embed fix)' [production]
02:46 <LocalisationUpdate> ResourceLoader cache refresh completed at 2014-01-18 02:46:09+00:00 [production]
02:26 <LocalisationUpdate> completed (1.23wmf11) at 2014-01-18 02:26:05+00:00 [production]
02:13 <LocalisationUpdate> completed (1.23wmf10) at 2014-01-18 02:12:58+00:00 [production]
01:28 <LeslieCarr> Leslie Carr is signing off. au revoir! [production]
01:23 <Reedy> [01:18:21] <LeslieCarr> ok !log exporting no routes over XO transit in sdtpa [production]
2014-01-17 §
22:12 <awight> updated legacy paypal a4748700afaa4a7c5e9b84bfa1f78e3298e04ad1 to 725055ac1e2217806245c595d90b1ab63ec49007 [production]
21:52 <ori> to restart morebots: ssh to tool labs, become morebots, then: qdel $(qstat | grep production | cut -d' ' -f 1) ; sleep 5 ; jstart -N production /usr/lib/adminbot/adminlogbot.py --config ./confs/production-logbot.py [production]
21:51 <ori> restarted morebots [production]
21:51 <LeslieCarr> removed myself from all mail aliases [production]
16:14 <hashar> synchronized wmf-config/InitialiseSettings.php 'touch, restore $oaiAgentRegex' [production]