51-100 of 10000 results (44ms)
2018-01-02 §
12:21 <akosiaris> empty ganeti1008 for kernel downgrade. T181121 [production]
12:11 <jynus> add missing mysql grants to db1055 and db1056 [production]
11:42 <moritzm> installing ncurses security updates [production]
11:39 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Revert: Repool db1055 & db1056 as x1 replicas (duration: 00m 51s) [production]
11:32 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 50s) [production]
11:31 <mobrovac@tin> Finished deploy [citoid/deploy@ee0bdf4]: Update to service template v0.5.4 - T151394 (duration: 04m 19s) [production]
11:30 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 50s) [production]
11:27 <mobrovac@tin> Started deploy [citoid/deploy@ee0bdf4]: Update to service template v0.5.4 - T151394 [production]
09:45 <oblivian@puppetmaster1001> conftool action : set/pooled=inactive; selector: cluster=api_appserver,name=mw1(1.*|200).eqiad.wmnet [production]
09:44 <_joe_> setting api_appservers in the mw1180-1200 range to pooled=inactive, T183895 [production]
09:43 <oblivian@puppetmaster1001> conftool action : set/pooled=inactive; selector: cluster=appserver,name=mw1(1.*|200).eqiad.wmnet [production]
09:37 <_joe_> setting appservers in the mw1180-1200 range to pooled=inactive, T183895 [production]
09:28 <godog> reboot ms-be1033 - T183724 [production]
08:52 <_joe_> restarting also mw1226-8, mw1223, mw1201,mw1203, mw1205-7 [production]
08:36 <_joe_> likewise for mw1285,mw1235,mw1232 [production]
08:29 <_joe_> restarting hhvm on mw1280,1282 for the same reasons [production]
08:26 <_joe_> restarting hhvm on mw1317, multiple threads stuck in HPHP::jit::enterTCImpl [production]
08:23 <elukey> restart druid coordinators on druid* to pick up new jvm settings [production]
08:19 <_joe_> restarting hhvm on mw1313, concurrency HPHP::VariableUnserializer::unserializeVariant [production]
08:06 <marostegui> Deploy alter table on db1039 (already depooled) - T174569 [production]
07:56 <marostegui> Deploy schema change on dbstore1001.s7 - T174569 [production]
06:51 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1110 (duration: 00m 52s) [production]
06:42 <marostegui> Stop db1110 and dbstore1002.s5 replication in sync [production]
06:28 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1110 to reimport dewiki.langlinks on dbstore1002 (duration: 00m 50s) [production]
2018-01-01 §
22:45 <eileen1> re-enable silverpop jobs process-control config revision is 4a4c61ae1b [production]
22:36 <eileen1> update process-control to process-control config revision is 65b65972a9 (silverpop password change) [production]
21:01 <addshore@tin> Synchronized php-1.31.0-wmf.12/extensions/WikimediaEvents: [[gerrit:399627|Add onBeforeInitializeWMDECampaign]] T182797 T182794 PT 2/2 (duration: 00m 53s) [production]
20:59 <addshore@tin> Synchronized php-1.31.0-wmf.12/extensions/WikimediaEvents/WikimediaEventsHooks.php: [[gerrit:399627|Add onBeforeInitializeWMDECampaign]] T182797 T182794 PT 1/2 (duration: 00m 55s) [production]
2017-12-30 §
18:22 <ejegg> disabled failing silverpop data fetch jobs [production]
2017-12-29 §
18:31 <mutante> gerrit: restart service to apply change from Dec 20th, avoid logspam due to unapplied config change [production]
15:38 <ariel@tin> Finished deploy [dumps/dumps@51d4fd6]: enable pageslogging dump in parallel jobs (duration: 00m 02s) [production]
15:38 <ariel@tin> Started deploy [dumps/dumps@51d4fd6]: enable pageslogging dump in parallel jobs [production]
14:46 <apergos> restarted puppetdb on nitrogen, puppetdb refusing to hand out facts again [production]
08:34 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1110 (duration: 00m 52s) [production]
08:19 <marostegui> Stop replication in sync on dbstore1002.s5 and db1110 [production]
08:03 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1110 to reimport dewiki.imagelinks on dbstore1002 (duration: 00m 52s) [production]
07:43 <marostegui> Fixing dbstore1002 dewiki.pagelinks [production]
2017-12-28 §
14:18 <marostegui> Power cycle pc2005 as it is down [production]
2017-12-27 §
19:54 <godog> power reset ms-be1033 [production]
19:47 <godog> reboot ms-be1033 - T183724 [production]
17:00 <twentyafterfour> restarting apache on phab1001 [production]
15:54 <mobrovac@tin> Started restart [electron-render/deploy@94d27d7]: Bounce Electron, stuck - T174916 [production]
2017-12-26 §
16:33 <mobrovac@tin> Finished deploy [mathoid/deploy@63b2ddc]: Bring back Mathoid in codfw to v0.6.5 in sync with eqiad - T179419 T172767 (duration: 01m 44s) [production]
16:31 <mobrovac@tin> Started deploy [mathoid/deploy@63b2ddc]: Bring back Mathoid in codfw to v0.6.5 in sync with eqiad - T179419 T172767 [production]
16:27 <mutante> re-generated DNS zones to add new language lfn (Lingua Franca Nova) (T183561) [production]
12:00 <Amir1> ladsgroup@terbium:~$ mwscript extensions/Cognate/maintenance/populateCognatePages.php --wiki hifwiktionary (T180785) [production]
11:59 <Amir1> ladsgroup@terbium:~$ mwscript extensions/Cognate/maintenance/populateCognateSites.php --wiki aawiktionary --site-group wiktionary (T180785) [production]
11:05 <ariel@tin> Finished deploy [dumps/dumps@8c44d0a]: re-enable file size reporting for dump files still being written (duration: 00m 02s) [production]
11:05 <ariel@tin> Started deploy [dumps/dumps@8c44d0a]: re-enable file size reporting for dump files still being written [production]
08:34 <akosiaris> migrate logstash1007 off ganeti1006 T181121 [production]