5551-5600 of 10000 results (63ms)
2018-01-22 §
10:31 <arturo> aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent -t -v' <--- check again how is the cluster with puppet [tools]
10:18 <arturo> T181948 deploy misctools 1.27 in the cluster [tools]
09:51 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1099:3311 - T162807 (duration: 00m 56s) [production]
09:50 <jynus> running heavy reads on db2043, db2036 to try to reproduce s3 codfw crash [production]
09:25 <marostegui> Stop replication in sync db1099:3311 and db1089 - T162807 [production]
09:21 <marostegui> Stop MySQL on db1030 to clone db1063 - T184397 [production]
09:20 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1030 - T184397 (duration: 00m 56s) [production]
09:13 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1066, depool db1099:3311 - T162807 (duration: 00m 56s) [production]
08:41 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Increase db1067 weight (duration: 00m 56s) [production]
08:31 <moritzm> upgrading video scalers to HHVM 3.18.7 [production]
07:51 <marostegui> Stop replication in sync db1089 and db1066 - T162807 [production]
07:46 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1067, depool db1066 - T162807 (duration: 00m 56s) [production]
07:11 <marostegui> Stop replication in sync db1089 and db1067 - T162807 [production]
07:09 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1089 and db1067 - T162807 (duration: 00m 56s) [production]
07:04 <elukey> truncated /var/log/upstart/neutron-server.log on labtestnet2001 - / disk space exhausted [production]
06:41 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Move db1063 from s8 to s6 - T184397 (duration: 00m 58s) [production]
06:18 <marostegui> Compress ruwiki on db1102 - T182450 [production]
02:37 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.17) (duration: 07m 24s) [production]
2018-01-21 §
17:21 <marostegui> Compress frwiki and jawiki on db1102 - T182450 [production]
12:03 <marostegui> Defragment s2 on db1102 - T182450 [production]
02:35 <urandom> bootstrapping restbase2012-b - T184100 [production]
2018-01-20 §
23:20 <urandom> bootstrapping restbase2012-a - T184100 [production]
17:36 <elukey> forced bbu learn cycle on analytics1038 (cache policy flapping from WriteBack to WriteThrough) [production]
16:57 <urandom> bootstrapping restbase2011-c - T184100 [production]
12:53 <urandom> bootstrapping restbase2011-b - T184100 [production]
09:09 <legoktm> reloading zuul to deploy https://gerrit.wikimedia.org/r/405404 https://gerrit.wikimedia.org/r/405352 https://gerrit.wikimedia.org/r/404972 https://gerrit.wikimedia.org/r/404229 https://gerrit.wikimedia.org/r/404230 https://gerrit.wikimedia.org/r/354667 [releng]
03:32 <urandom> bootstrapping restbase2011-a - T184100 [production]
2018-01-19 §
22:53 <matt_flaschen> Ran (time foreachwikiindblist flow.dblist extensions/Flow/maintenance/FlowFixInconsistentBoards.php --force) 2>&1|tee --append ~/FlowFixInconsistentBoards_all_2018-01-19_actual_force.txt [production]
21:28 <urandom> bootstrapping restbase2010-c - T184100 [production]
20:35 <wikibugs> Updated channels.yaml to: e85e5b9d9cd2b574d72bf2292b19f8a256f888fd Follow-up: I forgot Phabricator's sub-projects [tools.wikibugs]
19:43 <mutante> ms-be3003 - power up via mgmt to check if still connected and usable as temp bastion (T184936) [production]
19:16 <wikibugs> Updated channels.yaml to: 82f35a133ffe502920845f25a616e3e64de66be2 Remove #-devtools, merge into #-releng [tools.wikibugs]
18:58 <urandom> bootstrapping restbase2010-b - T184100 [production]
17:58 <chasemp> labcontrol1002:~# ip addr del 208.80.154.12/32 dev eth0 [production]
17:58 <chasemp> labcontrol1002:~# ip addr del 208.80.154.102/32 dev eth0 [production]
17:55 <chasemp> labcontrol1001:~# ip addr del 208.80.154.94/32 dev eth0 [production]
17:50 <reedy@tin> Synchronized dblists/s3.dblist: alphasort and remove dupes (duration: 01m 01s) [production]
17:32 <arturo> T185314 deploying new version of jobutils 1.27 [tools]
17:23 <zeljkof> Reloading Zuul to deploy 25cab6fdb53f1e5d284192159c43f0ce28f6e604 [releng]
17:11 <jynus> stopping mariadb on db2016,17,18,19,23,28&29 T184090 [production]
17:10 <zeljkof> Reloading Zuul to deploy 7d6b4eecc6d7854de7db6714bcfd006d860f3e28 [releng]
16:27 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool ddb1089 and depool db1067 (duration: 00m 56s) [production]
16:10 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 - T162807 (duration: 00m 56s) [production]
16:01 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1109 - T174569 (duration: 00m 56s) [production]
15:57 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool db1089 - T162807 (duration: 00m 56s) [production]
15:43 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Tune s1 and s3 database weights (duration: 00m 57s) [production]
15:38 <anomie> Running migrateArchiveText.php on all wikis that need it (T184629) [production]
15:34 <elukey> added deployment-eventlog02.deployment-prep.eqiad.wmflabs to /etc/ssh/ssh_known_hosts on deployment-tin (following https://phabricator.wikimedia.org/T116206#2251441) to unblock "Host key verification failed" for Analytics [releng]
15:24 <anomie> Running migrateArchiveText.php on metawiki (T184629) [production]
15:23 <godog> bootstrap cassandra-a on restbase2010 - T184100 [production]