5551-5600 of 10000 results (57ms)
2017-09-05 §
06:29 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1083 - T174509 (duration: 00m 46s) [production]
06:25 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1089 - T174509 (duration: 00m 46s) [production]
06:09 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1089 - T174509 (duration: 00m 55s) [production]
02:30 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Sep 5 02:30:03 UTC 2017 (duration 6m 37s) [production]
02:23 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.16) (duration: 07m 28s) [production]
2017-09-04 §
23:32 <Dereckson> Set e-mail to Hexabot bot account to allow account recovery (T174973) [production]
22:07 <legoktm> starting script to reparse all pages in parsoid for Linter (python2 parsoid-reparse.py http://parsoid.discovery.wmnet:8000 --sitematrix --linter-only --skip-closed https://aa.wikipedia.org/w/api.php) - T161556 [production]
17:50 <moritzm> uploaded debdeploy 0.0.99-2 to apt.wikimedia.org [production]
17:19 <gehel@tin> Finished deploy [wdqs/wdqs@1caaa30]: weekly WDQS deployment (duration: 02m 46s) [production]
17:17 <gehel@tin> Started deploy [wdqs/wdqs@1caaa30]: weekly WDQS deployment [production]
16:29 <mobrovac@tin> Finished deploy [restbase/deploy@8c9c436]: (no justification provided) (duration: 06m 23s) [production]
16:23 <mobrovac@tin> Started deploy [restbase/deploy@8c9c436]: (no justification provided) [production]
16:10 <mobrovac> restbase depooled restbase10(0[89]|10) and restbase2005 for T169939 [production]
16:07 <_joe_> rolling restart of pybal in codfw/eqiad low-traffic pools for picking up the new jobrunner LVS endpoint [production]
15:42 <oblivian@puppetmaster1001> conftool action : set/pooled=yes; selector: cluster=jobrunner,service=nginx [production]
14:51 <godog> reimage restbase2003 for use with cassandra 3 / jbod - T169939 [production]
14:27 <moritzm> enabled LDAP authentication for ganglia.wikimedia.org [production]
13:37 <gehel> rolling restart of elasticsearch/logstash to cleanup unused plugins - T174933 [production]
13:34 <gehel> delete leftover grafana-dashboards index on elasticsearch logstash cluster [production]
13:22 <gehel> restart elasticsearch on logstash1006 to test plugin removal - T174933 [production]
12:26 <moritzm> uploaded debdeploy 0.0.99-1+trusty/trusty-wikimedia to apt.wikimedia.org [production]
12:18 <marostegui> Stop MySQL on db1037 as it is going to be decommissioned - T174902 [production]
12:10 <elukey> rolling restart of zookeeper on conf100[123] for jvm security updates [production]
11:53 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1037 as it will be decommissioned - T174902 (duration: 00m 46s) [production]
11:52 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1037 as it will be decommissioned - T174902 (duration: 00m 46s) [production]
11:45 <godog> restart varnish on cp1099 - mailbox lag [production]
10:22 <ariel@tin> Finished deploy [dumps/dumps@0b32eef]: fix abstract dumps variants regression (duration: 00m 02s) [production]
10:22 <ariel@tin> Started deploy [dumps/dumps@0b32eef]: fix abstract dumps variants regression [production]
09:51 <moritzm> uploaded debdeploy 0.0.99-1+jessie/jessie-wikimedia to apt.wikimedia.org [production]
09:41 <mobrovac@tin> Started restart [mathoid/deploy@44ea6d8]: Restart for libxml update [production]
09:36 <moritzm> uploaded debdeploy 0.0.99-1/stretch-wikimedia to apt.wikimedia.org [production]
08:52 <elukey> restart zookeeper on conf2003 for jvm security updates [production]
08:48 <marostegui> Stop MySQL on db1045 as it will be decommissioned - T174806 [production]
08:47 <moritzm> installing mercurial security updates [production]
08:40 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1045 as it will be decommissioned - T174806 (duration: 00m 46s) [production]
08:39 <moritzm> installing libgcrypt20 security updates [production]
08:39 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1045 as it will be decommissioned - T174806 (duration: 00m 48s) [production]
08:33 <godog> upload scap 3.7.0-1 to apt.w.o - T127762 [production]
07:39 <moritzm> installing gnupg security updates [production]
07:03 <marostegui> Deploy alter table on s3 codfw master with replication on kpbwiki - T168661 [production]
07:02 <_joe_> starting additional runJobs instance for htmlcacheupdate on commons T173710 [production]
04:52 <marostegui> Deploy alter table on db1052 - T168661 [production]
02:44 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Sep 4 02:44:15 UTC 2017 (duration 6m 52s) [production]
02:37 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.16) (duration: 08m 17s) [production]
2017-09-03 §
17:52 <elukey> depooled cp4024 (ulsfo upload) due to kernel errors in dmesg [production]
15:57 <ema> restart varnish backend on cp1073 (mailbox lag) [production]
09:41 <ema@neodymium> conftool action : set/pooled=yes; selector: name=cp4024.ulsfo.wmnet,service=varnish-fe [production]
09:41 <ema@neodymium> conftool action : set/pooled=yes; selector: name=cp4024.ulsfo.wmnet,service=nginx [production]
09:25 <ema> cp4024 is down, powercycled [production]
09:13 <_joe_> manually fixed value of /conftool/v1/pools/eqiad/api_appserver/apache2/mw1208.eqiad.wmnet in codfw to allow replication to restart [production]