701-750 of 10000 results (48ms)
2017-09-04 §
14:27 <moritzm> enabled LDAP authentication for ganglia.wikimedia.org [production]
13:37 <gehel> rolling restart of elasticsearch/logstash to cleanup unused plugins - T174933 [production]
13:34 <gehel> delete leftover grafana-dashboards index on elasticsearch logstash cluster [production]
13:22 <gehel> restart elasticsearch on logstash1006 to test plugin removal - T174933 [production]
12:26 <moritzm> uploaded debdeploy 0.0.99-1+trusty/trusty-wikimedia to apt.wikimedia.org [production]
12:18 <marostegui> Stop MySQL on db1037 as it is going to be decommissioned - T174902 [production]
12:10 <elukey> rolling restart of zookeeper on conf100[123] for jvm security updates [production]
11:53 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1037 as it will be decommissioned - T174902 (duration: 00m 46s) [production]
11:52 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1037 as it will be decommissioned - T174902 (duration: 00m 46s) [production]
11:45 <godog> restart varnish on cp1099 - mailbox lag [production]
10:22 <ariel@tin> Finished deploy [dumps/dumps@0b32eef]: fix abstract dumps variants regression (duration: 00m 02s) [production]
10:22 <ariel@tin> Started deploy [dumps/dumps@0b32eef]: fix abstract dumps variants regression [production]
09:51 <moritzm> uploaded debdeploy 0.0.99-1+jessie/jessie-wikimedia to apt.wikimedia.org [production]
09:41 <mobrovac@tin> Started restart [mathoid/deploy@44ea6d8]: Restart for libxml update [production]
09:36 <moritzm> uploaded debdeploy 0.0.99-1/stretch-wikimedia to apt.wikimedia.org [production]
08:52 <elukey> restart zookeeper on conf2003 for jvm security updates [production]
08:48 <marostegui> Stop MySQL on db1045 as it will be decommissioned - T174806 [production]
08:47 <moritzm> installing mercurial security updates [production]
08:40 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1045 as it will be decommissioned - T174806 (duration: 00m 46s) [production]
08:39 <moritzm> installing libgcrypt20 security updates [production]
08:39 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1045 as it will be decommissioned - T174806 (duration: 00m 48s) [production]
08:33 <godog> upload scap 3.7.0-1 to apt.w.o - T127762 [production]
07:39 <moritzm> installing gnupg security updates [production]
07:03 <marostegui> Deploy alter table on s3 codfw master with replication on kpbwiki - T168661 [production]
07:02 <_joe_> starting additional runJobs instance for htmlcacheupdate on commons T173710 [production]
04:52 <marostegui> Deploy alter table on db1052 - T168661 [production]
02:44 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Sep 4 02:44:15 UTC 2017 (duration 6m 52s) [production]
02:37 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.16) (duration: 08m 17s) [production]
2017-09-03 §
17:52 <elukey> depooled cp4024 (ulsfo upload) due to kernel errors in dmesg [production]
15:57 <ema> restart varnish backend on cp1073 (mailbox lag) [production]
09:41 <ema@neodymium> conftool action : set/pooled=yes; selector: name=cp4024.ulsfo.wmnet,service=varnish-fe [production]
09:41 <ema@neodymium> conftool action : set/pooled=yes; selector: name=cp4024.ulsfo.wmnet,service=nginx [production]
09:25 <ema> cp4024 is down, powercycled [production]
09:13 <_joe_> manually fixed value of /conftool/v1/pools/eqiad/api_appserver/apache2/mw1208.eqiad.wmnet in codfw to allow replication to restart [production]
00:59 <legoktm> legoktm@terbium:~$ mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php --wiki=hiwikiversity --backend=local-multiwrite # T174859 [production]
2017-09-01 §
16:18 <bblack> restart varnish backend on cp1074 (mailbox lag) [production]
15:15 <urandom> Restarting Cassandra: restbase-dev100[5-6]-{a,b} [production]
15:06 <urandom> Restarting Cassandra: restbase-dev1004-{a,b} [production]
15:00 <reedy@tin> Synchronized php-1.30.0-wmf.16/extensions/Popups/: T174724 (duration: 00m 46s) [production]
14:28 <marostegui> Add 150G to /srv partition on labsdb1001 [production]
13:42 <marostegui> Rename table pr_index on enwiki on db1089 - T174782 [production]
13:10 <marostegui> Stop MySQL on db1026 as it will be decommissioned - T174763 [production]
12:32 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1081 original weight - T168661 (duration: 00m 43s) [production]
12:19 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1026 as it will be decommissioned - T174763 (duration: 00m 43s) [production]
12:17 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1026 as it will be decommissioned - T174763 (duration: 00m 43s) [production]
11:50 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1081 weight - T168661 (duration: 00m 43s) [production]
11:23 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1081 with low weight - T168661 (duration: 00m 44s) [production]
10:35 <elukey> stop puppet on thorium and disable root rsyncs - T174756 [production]
10:31 <marostegui> Upgrade MariaDB to 10.0.32 on db1081 - T168661 [production]
10:29 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1081 - T168661 (duration: 00m 43s) [production]