2351-2400 of 10000 results (17ms)
2020-10-12 §
15:22 <hashar> Updated non voting codehealth jobs to Java11 for T259646: mwcore-codehealth-master-non-voting mwext-codehealth-master-non-voting [releng]
14:44 <_joe_> freed 1.5 GB of space on ms-be2036 by running "apt-get clean" [production]
14:05 <moritzm> uploaded php7.2 7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1+icu63 to component/icu63 T264991 [production]
13:58 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/maven-java11:1.0.0 # T259646 [releng]
13:52 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/maven:3.5.2-1 [releng]
13:45 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/maven-java8:0.7.0 [releng]
13:35 <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/java11:0.1 [releng]
12:39 <moritzm> installing rails security updates on Stretch [production]
12:26 <moritzm> installing spice security updates on Buster [production]
11:38 <Urbanecm> EU B&C done [production]
11:36 <joal> Clean druid test-datasources [analytics]
11:32 <elukey> remove analytics-meta lvm backup settings from an-coord1001 [analytics]
11:32 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: fff2532424f84970962f7de1e35d4250b83cb3da: [testwiki, test2wiki] Allow bureaucrats to grant import rights (duration: 00m 58s) [production]
11:28 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 4966e8a6b8ae4e6d5623dd35e65ed8fcf3338bc1: Enable wgCheckUserLogLogins at all wikis but few large wikis (T253802) (duration: 00m 58s) [production]
11:27 <hnowlan@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) [production]
11:23 <elukey> remove analytics-meta lvm backup settings from an-master1002 [analytics]
11:18 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:631809|Require autoconfirmed status to edit Wikidata Properties (T254280)]] (duration: 01m 00s) [production]
10:26 <hnowlan@cumin1001> START - Cookbook sre.cassandra.roll-restart [production]
10:26 <hnowlan> roll-restarting restbase201[345678] for cert refresh [production]
08:50 <moritzm> uploaded libxml2 2.9.4+dfsg1-2.2+deb9u3+wmf1 to component/icu63 T264991 [production]
08:42 <hashar> integration-castor: nuking large /srv/jenkins-workspace/caches/wikidata-query-rdf/master/wikidata-query-rdf-maven-java8-docker [releng]
08:39 <hashar> integration-castor: nuking 17GB large /srv/jenkins-workspace/caches/wikidata-query-rdf/master/wikidata-query-rdf-maven-java8-docker-site-publish [releng]
07:54 <godog> reboot ms-be2036 - T265208 [production]
07:53 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:53 <filippo@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:53 <filippo@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
07:53 <filippo@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:02 <elukey> reduce hdfs block replication factor on Hadoop test to 2 [analytics]
05:37 <elukey> decom analytics1047 from the Hadoop cluster [analytics]
2020-10-11 §
08:33 <elukey> drop some old namenode backups under /srv on an-master1002 to free some space [analytics]
08:24 <elukey> decommission analytics1046 from the hadoop cluster [analytics]
08:12 <elukey> clean up logs on an-launcher1002 (disk space full) [analytics]
2020-10-10 §
17:07 <bstorm> cleared errors on tools-sgeexec-0912.tools.eqiad.wmflabs to get the queue moving again [tools]
12:01 <elukey> decommission analytics1045 from the Hadoop cluster [analytics]
01:32 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:633281|Enable session-ip log channel everywhere (T264799)]] (duration: 00m 59s) [production]
00:54 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:633277|Enable session-ip log channel on all but enwiki (T264799)]] (duration: 01m 01s) [production]
00:18 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:633276|Enable session-ip log channel on eswiki (T264799)]] (duration: 00m 55s) [production]
00:13 <mutante> built prometheus-nutcracker-exporter for buster and imported on apt1001 (0.2+nmu1) [production]
2020-10-09 §
23:44 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:633274|Enable session-ip log channel on Wikidata (T264799)]] (duration: 00m 59s) [production]
23:25 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:633272|Enable session-ip log channel on Commons (T264799)]] (duration: 00m 59s) [production]
23:13 <mutante> maps2010 is down since almost 3 days - unhandled crit alert but nothing in SAL and only related ticket says resolved - powercycling it - boots normal but doesn't have a prod role (T260271) [production]
23:07 <mutante> maps2010 is down since almost 3 days - unhandled crit alert but nothing in SAL or tickets [production]
23:06 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
23:06 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:06 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:06 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:06 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:06 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:03 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:03 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]