3251-3300 of 10000 results (46ms)
2016-01-18 §
14:24 <godog> powercycle praseodymium [production]
10:42 <godog> powercycle ms-be2016, high load avg [production]
10:16 <godog> dist-upgrade ms-be3002 to trusty [production]
02:57 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Mon Jan 18 02:57:41 UTC 2016 (duration 7m 8s) [production]
02:50 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 08m 39s) [production]
02:49 <YuviPanda> updated annualreport for foks [production]
02:30 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 11m 38s) [production]
01:07 <ottomata> restarted eventlogging again. A single raw client side processor consumer seemed stuck (according to burrow). seeing offset commit errors in logs. [analytics]
2016-01-17 §
18:02 <legoktm> deploying https://gerrit.wikimedia.org/r/264605 [releng]
08:26 <ottomata> restarting eventlogging to see if it'll help burrow reported kafka consumer lag [analytics]
04:58 <YuviPanda> started restbase on restbase1002 [production]
02:53 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Sun Jan 17 02:53:19 UTC 2016 (duration 6m 59s) [production]
02:46 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 08m 53s) [production]
02:26 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 41s) [production]
01:47 <paravoid> restarting HHVM on mw1120, mw1125, mw1127, mw1132, mw1148; OOM [production]
2016-01-16 §
21:47 <legoktm> deploying https://gerrit.wikimedia.org/r/264489 [releng]
21:36 <legoktm> deploying https://gerrit.wikimedia.org/r/264488 [releng]
21:29 <legoktm> deploying https://gerrit.wikimedia.org/r/264487 [releng]
21:21 <legoktm> deploying https://gerrit.wikimedia.org/r/264483 https://gerrit.wikimedia.org/r/264485 [releng]
20:58 <legoktm> deploying https://gerrit.wikimedia.org/r/264492 [releng]
19:52 <andrewbogott> renaming and reimaging labcontrol2001 -> labtestweb2001 [production]
18:55 <jzerebecki> reloadin zuul for 996c558..5f8eb50 [releng]
15:57 <milimetric> piwik is taking events on bohrium but the interface can't complete the queries to load because there's too much data. Mysql is maxing the CPU but it seems ok for now, will check again Monday. [production]
15:22 <milimetric> restarted mysql on bohrium because it had stopped working (probably due to piwik performance problems) [production]
09:12 <legoktm> deploying https://gerrit.wikimedia.org/r/264448 [releng]
09:01 <legoktm> deploying https://gerrit.wikimedia.org/r/264446 and https://gerrit.wikimedia.org/r/264447 [releng]
07:46 <legoktm> sudo -u jenkins-deploy mv /mnt/jenkins-workspace/workspace/mediawiki-core-phplint /mnt/jenkins-workspace/workspace/mediawiki-core-php53lint on all precise slaves [releng]
07:17 <legoktm> deploying https://gerrit.wikimedia.org/r/264444 [releng]
06:31 <legoktm> deploying https://gerrit.wikimedia.org/r/264441 [releng]
06:10 <legoktm> added phpflavor-php53 label to all phpflavor-zend slaves [releng]
04:50 <ircnotifier> legoktm: Deployed 9fde1aa8a06e27a8a08f5d2468cc5e00799dd43f Bump redis delay to 2 seconds to avoid flooding wb2-phab [tools.wikibugs]
03:02 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Sat Jan 16 03:02:21 UTC 2016 (duration 6m 57s) [production]
02:55 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 08m 35s) [production]
02:35 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 18m 55s) [production]
2016-01-15 §
22:43 <aaron@tin> aaron@tin Synchronized wmf-config/CommonSettings.php: Set $wgCentralAuthUseSlaves for testwiki (duration: 00m 33s) [production]
22:38 <mutante> gadolinium - shutdown -h now [production]
22:35 <mutante> erbium - killing from puppet/icinga/salt [production]
21:54 <mutante> mira - starting salt [production]
21:29 <mutante> protactinium - shut down, unused system with outdated software [production]
21:28 <milimetric> importing db data to new prod server [wikimetrics]
21:28 <milimetric> set up symlinks for static files on new prod server [wikimetrics]
21:27 <milimetric> stopping old prod queue and scheduler [wikimetrics]
21:09 <mutante> (ganglia for ulsfo will be affected, brb) [production]
21:07 <mutante> bast4001 - reinstalling with jessie [production]
18:54 <ori> disabled gzip in apache for javascript mime types and did an apache config reload [production]
18:34 <chasemp> tools-mail-01 is locked up I am rebooting [tools]
18:04 <ori@tin> ori@tin Synchronized docroot and w: Ie60638b0: Mirror homepage.js from 15.wikipedia.org (duration: 00m 42s) [production]
16:01 <godog> bounce hhvm on mw1129 / mw1204 [production]
15:41 <godog> reimage ms-be3001 with trusty [production]
14:54 <godog> reimage ms-fe3002 with trusty [production]