51-100 of 10000 results (27ms)
2016-01-17 §
02:26 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 10m 41s) [production]
01:47 <paravoid> restarting HHVM on mw1120, mw1125, mw1127, mw1132, mw1148; OOM [production]
2016-01-16 §
19:52 <andrewbogott> renaming and reimaging labcontrol2001 -> labtestweb2001 [production]
15:57 <milimetric> piwik is taking events on bohrium but the interface can't complete the queries to load because there's too much data. Mysql is maxing the CPU but it seems ok for now, will check again Monday. [production]
15:22 <milimetric> restarted mysql on bohrium because it had stopped working (probably due to piwik performance problems) [production]
03:02 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Sat Jan 16 03:02:21 UTC 2016 (duration 6m 57s) [production]
02:55 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 08m 35s) [production]
02:35 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 18m 55s) [production]
2016-01-15 §
22:43 <aaron@tin> aaron@tin Synchronized wmf-config/CommonSettings.php: Set $wgCentralAuthUseSlaves for testwiki (duration: 00m 33s) [production]
22:38 <mutante> gadolinium - shutdown -h now [production]
22:35 <mutante> erbium - killing from puppet/icinga/salt [production]
21:54 <mutante> mira - starting salt [production]
21:29 <mutante> protactinium - shut down, unused system with outdated software [production]
21:09 <mutante> (ganglia for ulsfo will be affected, brb) [production]
21:07 <mutante> bast4001 - reinstalling with jessie [production]
18:54 <ori> disabled gzip in apache for javascript mime types and did an apache config reload [production]
18:04 <ori@tin> ori@tin Synchronized docroot and w: Ie60638b0: Mirror homepage.js from 15.wikipedia.org (duration: 00m 42s) [production]
16:01 <godog> bounce hhvm on mw1129 / mw1204 [production]
15:41 <godog> reimage ms-be3001 with trusty [production]
14:54 <godog> reimage ms-fe3002 with trusty [production]
14:13 <mark> Temporarily paused md126 RAID check on labstore1001 (sync_action idle) [production]
14:09 <chasemp> phab restart phd (reports as not running in phab itself) seems ok now [production]
14:03 <mark> set sync_speed_min to 5000 for md126 on labstore1001 [production]
13:28 <demon@tin> demon@tin Synchronized wmf-config/InitialiseSettings.php: w:he as import source for commonswiki (duration: 00m 49s) [production]
12:17 <hashar> restarting Jenkins for plugins updates [production]
11:07 <_joe_> re-enabled puppet on mw1013, restarted HHVM to make it pick up our latest changes [production]
10:01 <moritzm> installed ganeti security updates [production]
09:18 <moritzm> installed git security updates on all jessie systems [production]
03:10 <l10nupdate@tin> l10nupdate@tin ResourceLoader cache refresh completed at Fri Jan 15 03:10:09 UTC 2016 (duration 6m 48s) [production]
03:03 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 16m 02s) [production]
02:30 <krenair@tin> krenair@tin Synchronized php-1.27.0-wmf.10/includes/api/ApiQueryRecentChanges.php: https://gerrit.wikimedia.org/r/264231 (duration: 00m 42s) [production]
02:29 <mwdeploy@tin> mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 14m 00s) [production]
02:23 <YuviPanda> pull annualreport git repo on bromine for Krenair [production]
01:00 <krenair@tin> krenair@tin Synchronized php-1.27.0-wmf.10/includes/api/ApiQueryWatchlist.php: https://gerrit.wikimedia.org/r/#/c/264224/ (duration: 00m 31s) [production]
00:27 <krenair@tin> krenair@tin Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/263905/ (duration: 00m 32s) [production]
00:24 <krenair@tin> krenair@tin Synchronized wmf-config/InitialiseSettings.php: touch (duration: 00m 31s) [production]
00:22 <krenair@tin> krenair@tin Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/264091/ (duration: 00m 32s) [production]
00:06 <mobrovac> restbase started a dump of enwiki to populate storage with mobileapps renders [production]
2016-01-14 §
23:56 <mobrovac> restbase end deploy of dac31a8c [production]
23:49 <mobrovac> restbase start deploy of dac31a8c [production]
22:17 <csteipp> deployed patch for T122807 [production]
19:55 <ottomata> restarted eventlogging_sync script to insert batches of 1000 [production]
19:31 <dduvall@tin> dduvall@tin rebuilt wikiversions.php and synchronized wikiversions files: rollback labswiki to wmf.9 [production]
19:02 <dduvall@tin> dduvall@tin rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.10 [production]
18:40 <bblack> removing old eqiad misc-web IP (DNS switched 50h ago (not 26 like above), TTLs are max 1h) [production]
18:39 <bblack> removing old eqiad misc-web IP (DNS switched 26h ago, TTLs are max 1h) [production]
18:01 <paravoid> turning up BGP with Zayo in eqiad [production]
16:25 <demon@tin> demon@tin Synchronized wmf-config/throttle.php: (no message) (duration: 00m 49s) [production]
15:48 <moritzm> installed DHCP security updates across the fleet [production]
14:44 <_joe_> powercycling mw1013, console stuck [production]