7151-7200 of 10000 results (36ms)
2016-04-19 §
01:34 <mutante> restbase2004 - starting crashed cassandra-b service [production]
01:11 <mutante> restbase2004 - unit cassandra-b is failed [production]
00:42 <mutante> kraz - signing puppet certs, adding salt keys [production]
00:29 <mutante> kraz.codfw.wmnet - initial install, adding to site [production]
2016-04-18 §
23:33 <ori@tin> Synchronized wmf-config/CommonSettings.php: I1547834: Remove unused ['bits'] configuration (duration: 00m 27s) [production]
23:22 <YuviPanda> running sync-common on mw1145 to catch up on deploys [production]
23:17 <YuviPanda> rebooted mw1145 from mgmt [production]
23:14 <ori@tin> Synchronized wmf-config: I0ec3c015f: Update parser cache configuration for tag-based hashing (duration: 01m 48s) [production]
23:08 <jynus@tin> Synchronized wmf-config/db-codfw.php: Depool pc2006 (duration: 01m 01s) [production]
23:06 <ori@tin> Synchronized php-1.27.0-wmf.21/includes/objectcache/SqlBagOStuff.php: Ie9799f5ea: Allow tag names for SqlBagOStuff consistent hashing (duration: 04m 08s) [production]
22:52 <krinkle@tin> Synchronized docroot/noc/conf: noc: remote bits references (duration: 00m 32s) [production]
22:45 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool pc2005 (duration: 00m 33s) [production]
22:44 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool pc1005 (duration: 00m 26s) [production]
22:20 <chasemp> resetting labcontrol1001 puppet master with auth.conf which is fixing all the puppet clients in Labs [production]
21:33 <gehel> deploying latest wdqs [production]
21:29 <papaul> conf200[1-3] - signing puppet certs, salt-key, initial run [production]
20:58 <papaul> OS install on conf200[1-3] [production]
20:55 <mutante> furud.codfw - initial puppet install, signing certs [production]
18:53 <bd808> Ran ~bd808/cleanup-var-crap.sh to free a bit of space on deployment-mediawiki01; this instance really needs a bigger disk [releng]
18:47 <bd808> deployment-mediawiki01: rm /var/log/apache2/*.gz [releng]
18:43 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool pc1005 (duration: 00m 28s) [production]
18:42 <jynus@tin> Synchronized wmf-config/db-codfw.php: Depool pc2005 (duration: 00m 26s) [production]
18:37 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool pc2004 (duration: 00m 25s) [production]
18:27 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool pc1004 (duration: 00m 36s) [production]
17:31 <elukey> restarted kafkatee on oxygen after module upgrade (was stuck from this morning due to a kafka broker restart) [production]
15:31 <andrewbogott> migration integration-slave-trusty-1018 to labvirt1009 [integration]
14:51 <jzerebecki> reloading zuul for d9e80bd..113ea21 [releng]
14:21 <ottomata> upgraded pykafka on hafnium and restarted statsv [production]
14:04 <andrewbogott> puppet agent -tv on labvirt1002 which restarted nrpe and resolved icinga alerts [production]
14:02 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool pc1004 (duration: 00m 29s) [production]
14:01 <jynus@tin> Synchronized wmf-config/db-codfw.php: Depool pc2004 (duration: 00m 27s) [production]
12:59 <jzerebecki> integration-slave-trusty-1014:~$ sudo -i rm -f /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/GlobalCssJs/.git/refs/heads/wmf/1.27.0-wmf.11.lock [releng]
12:57 <jzerebecki> integration-slave-trusty-1014:~$ sudo -i 'rm -f /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/GlobalCssJs/.git/refs/heads/wmf/1.27.0-wmf.11.lock' [releng]
11:30 <mobrovac> restbase restarting eqiad after https://gerrit.wikimedia.org/r/#/c/283958/ [production]
10:11 <elukey> execute sudo eventloggingctl restart on eventlogging1001 [analytics]
10:07 <_joe_> traffic from eqiad caches switched to codfw for restbase,citoid,cxserver [production]
09:43 <moritzm> installing openssh updates on jessie systems [production]
09:20 <_joe_> upgrading HHVM on terbium [production]
09:19 <godog> depool restbase100[56].eqiad.wmnet, about to get decomissioned [production]
09:17 <godog> repool restbase2006 after raid expansion [production]
09:08 <jynus@tin> Synchronized wmf-config/db-codfw.php: Update master comments in preparation for dc failover (duration: 00m 26s) [production]
09:07 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Update master comments in preparation for dc failover (duration: 01m 41s) [production]
07:02 <jynus> changing binlog_format to MIXED on db2018 [production]
2016-04-17 §
21:51 <urandom> Decommissioning restbase1005.eqiad.wmnet : T95253 [production]
21:50 <urandom> `systemctl mask cassandra' on restbase1006.eqiad.wmnet (node is decommissioned) : T95253 [production]
01:01 <legoktm> deploying https://gerrit.wikimedia.org/r/283837 [releng]
00:10 <urandom> Decommissioning restbase1006.eqiad.wmnet : T95253 [production]
2016-04-16 §
20:57 <bd808> Restarted to add bot to #wikimedia-codereview [tools.stashbot]
14:25 <andrewbogott> rebooting mw1132 — OOM [production]
14:24 <andrewbogott> rebooting mw1134 — OOM [production]