851-900 of 10000 results (28ms)
2015-12-02 §
08:38 <_joe_> planet1001 recovered as soon as I did get into console via gnt-instance [production]
06:25 <ori@tin> Synchronized php-1.27.0-wmf.7/extensions/NavigationTiming: Idb675cdce: Add isHiDPI and isHttp2 properties; drop isHttps (T119014) (duration: 00m 50s) [production]
05:53 <l10nupdate@tin> ResourceLoader cache refresh completed at Wed Dec 2 05:53:21 UTC 2015 (duration 53m 20s) [production]
02:27 <mutante|away> labcontrol2001 - disable puppet, kill from puppet stored configs [production]
02:26 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.7) (duration: 10m 47s) [production]
00:29 <mutante> puppetstoredconfigclean.rb labcontrol2001.wikimedia.org fixes icinga config [production]
2015-12-01 §
22:41 <mutante> stop/start ntp service on lvs2002 [production]
20:54 <mutante> re-enabled puppet on cache::misc, replaced planet cert [production]
20:19 <mutante> temp. disabling puppet on cache::misc [production]
18:14 <twentyafterfour> restarted apache on iridium to deploy https://phabricator.wikimedia.org/rPHEXd724c51a4144f7de548546875097da57ee1b38d7 [production]
17:05 <moritzm> restarting cassandra instances on restbase200[3-6] (to effect openjdk security updates and a few depending libs) [production]
17:00 <moritzm> restarting cassandra instances on restbase2002 (to effect openjdk security updates and a few depending libs) [production]
16:55 <moritzm> restarting cassandra instances on restbase2001 (to effect openjdk security updates and a few depending libs) [production]
16:22 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable VisualEditor for all accounts on eswiki. Part II [[gerrit:250474]] (duration: 00m 28s) [production]
16:21 <thcipriani@tin> Synchronized dblists/visualeditor-default.dblist: SWAT: Enable VisualEditor for all accounts on eswiki. Part I [[gerrit:250474]] (duration: 01m 29s) [production]
12:59 <godog> reimage mw2208 [production]
10:50 <moritzm> restarting on cassandra instances on restbase1008 (to effect openjdk security updates and a few depending libs) [production]
06:52 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Dec 1 06:52:40 UTC 2015 (duration 52m 39s) [production]
02:26 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.7) (duration: 10m 14s) [production]
02:18 <mutante> restarted hhvm on mw1147 [production]
01:47 <ori@tin> Synchronized docroot and w: (no message) (duration: 00m 30s) [production]
00:47 <catrope@tin> Synchronized php-1.27.0-wmf.7/extensions/VisualEditor: SWAT (duration: 00m 29s) [production]
00:47 <catrope@tin> Synchronized php-1.27.0-wmf.7/resources/src/mediawiki/mediawiki.ForeignStructuredUpload.js: SWAT (duration: 00m 30s) [production]
2015-11-30 §
21:55 <mutante> re-wrote l10nupdate cron; restarted cron service on tin [production]
20:05 <apergos> re-enabled puppet on neodymium, minion testing concluded for now [production]
19:47 <gwicke> running `nodetool decommission` on restbase1009 in preparation for the conversion to the multi-instance setup, per https://phabricator.wikimedia.org/T95253# [production]
19:31 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: rm deprecated/unused rate limit log config (duration: 00m 28s) [production]
17:27 <demon@tin> Synchronized php-1.27.0-wmf.7/extensions/WikimediaMaintenance/: need maint script errywhere (duration: 00m 28s) [production]
16:51 <thcipriani@tin> Synchronized php-1.27.0-wmf.7/extensions/ContentTranslation/modules/draft/ext.cx.draft.js: SWAT: Add some extra information to save failure logging [[gerrit:255956]] (duration: 00m 28s) [production]
16:38 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: Disable QuickSurveys reader segmentation survey [[gerrit:255448]] (duration: 00m 28s) [production]
16:30 <paravoid> mw1002 service hhvm restart [production]
16:17 <paravoid> rolling back to kernel 3.19 on lvs2001/2/3 [production]
15:29 <paravoid> stopping pybal on lvs2001/2/3 [production]
15:21 <paravoid> switching lvs2004/5/6 traffic back to lvs2001/2/3 [production]
15:13 <paravoid> switching lvs2001/2/3 traffic to lvs2004/5/6 and upgrading kernels [production]
15:12 <_joe_> restarting HHVM on mw1147 too, same reason as mw1114 [production]
15:10 <_joe_> restarting hhvm on mw1114, stuck in __pthread_cond_wait () [folly::EventBase::runInEventBaseThreadAndWait ()], apparently blocked in writing to stdout [production]
15:02 <paravoid> switching traffic from lvs4002 to lvs4004; upgrading lvs4002's kernel [production]
15:02 <paravoid> switching traffic back to lvs4001 [production]
14:57 <paravoid> switching traffic from lvs4001 to lvs4003; upgrading lvs4001's kernel [production]
14:45 <paravoid> switching traffic from lvs3001 to lvs3003; upgrading lvs3001's kernel [production]
14:38 <paravoid> switching traffic back to lvs3002 [production]
14:31 <paravoid> switching traffic from lvs3002 to lvs3004; upgrading lvs3002's kernel [production]
14:07 <bblack> upgrading varnishkafka package on all caches [production]
13:52 <bblack> updating varnishkafka on cp1065 [production]
11:03 <godog> upgrade python-statsd to 3.0.1 in eqiad [production]
10:59 <godog> upgrade python-statsd to 3.0.1 in codfw [production]
10:15 <godog> reenable puppet on graphite1001 [production]
10:10 <paravoid> re-enabling OSPF over cr2-eqiad:xe-5/2/2 <-> cr1-ulsfo:xe-0/0/3.538 [production]
10:09 <paravoid> re-enabling cr2-eqiad:xe-5/2/0 and xe-5/2/1 [production]