5301-5350 of 10000 results (70ms)
2019-05-21 §
13:20 <hashar> scap prep 1.34.0-wmf.6 # T220731 [production]
13:11 <hashar> Updated plugins on https://releases-jenkins.wikimedia.org/ [production]
13:09 <hashar> Restarting Jenkins T224002 [production]
12:45 <hashar> Cutting branch wmf/1.34.0-wmf.6 # T220731 [production]
12:22 <volans> restarting Icinga on icinga1001 to pick up new open files limits [production]
12:08 <jiji@deploy1001> Finished deploy [cpjobqueue/deploy@4588f16]: Migrating htmlCacheUpdate to PHP7 - T219148 (duration: 00m 54s) [production]
12:07 <jiji@deploy1001> Started deploy [cpjobqueue/deploy@4588f16]: Migrating htmlCacheUpdate to PHP7 - T219148 [production]
11:59 <mobrovac> started dewiki dumps - T215956 [production]
11:58 <mobrovac> started frwiki dumps - T215956 [production]
11:46 <mobrovac> started enwiki dumps - T215956 [production]
11:27 <Amir1> EU SWAT is done [production]
11:27 <ladsgroup@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:511658|Revert "Switch off php7 for investigation of production instabilities"]] (duration: 00m 50s) [production]
11:20 <volans> restarting Icinga on icinga2001 (passive server) to pick up new open file limits [production]
11:18 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:17 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:17 <jbond42> reboot wtp1025.eqiad.wmnet [production]
11:10 <ladsgroup@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:505816|Define wmgUseEntitySchema (T221651)]], part II (duration: 00m 49s) [production]
11:09 <mobrovac@deploy1001> Finished deploy [restbase/deploy@cf00120]: Switch Parsoid to simple k/v bucket - T215956 (duration: 25m 50s) [production]
11:08 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:505816|Define wmgUseEntitySchema (T221651)]], part I (duration: 00m 50s) [production]
11:07 <godog> swift codfw-prod: remove ms-be201[345] - T221068 [production]
10:59 <_joe_> rolling restart of php7.2-fpm across the fleet to pick up a config change [production]
10:43 <mobrovac@deploy1001> Started deploy [restbase/deploy@cf00120]: Switch Parsoid to simple k/v bucket - T215956 [production]
10:39 <jijiki> updating prometheus-mcrouter-exporter on mw* servers [production]
10:26 <godog> pool new restbase hosts - T219404 [production]
10:20 <filippo@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase1019.eqiad.wmnet [production]
09:49 <moritzm> updated buster netboot image to daily image from 20190521 [production]
09:26 <moritzm> reimaging graphite2001 to buster for some d-i tests [production]
08:58 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2104 as candidate master and as API (duration: 00m 51s) [production]
08:56 <marostegui> Stop MySQL on db2041 as it will be decommissioned T223950 [production]
06:59 <oblivian@deploy1001> Synchronized wmf-config/CommonSettings.php: Turning off php7 sampling for investigation in T223952 (duration: 00m 53s) [production]
06:55 <elukey> reboot of stat100[4,5,6,7] and notebook100[3,4] for kernel upgrades [production]
06:31 <marostegui> Stop mariadb on db2104 to convert it to s2 candidate master [production]
06:30 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2104 (duration: 00m 51s) [production]
05:50 <marostegui> Remove db2041 from tendril and zarcillo - T223950 [production]
05:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db2041 for decommissioning T223950 (duration: 00m 51s) [production]
05:42 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db2041 for decommissioning T223950 (duration: 00m 51s) [production]
05:16 <marostegui> Stop MySQL on db2040 [production]
05:16 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2040 (duration: 00m 50s) [production]
05:14 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Pool db2114 into s6 - T222772 (duration: 00m 50s) [production]
05:13 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Pool db2114 into s6 - T222772 (duration: 00m 51s) [production]
03:36 <urandom> bootstrapping restbase1027-c -- T219404 [production]
00:47 <urandom> bootstrapping restbase1027-b -- T219404 [production]
00:05 <aaron@deploy1001> Synchronized php-1.34.0-wmf.5/includes/libs/objectcache/APCUBagOStuff.php: 982299d635623279 (duration: 00m 54s) [production]
2019-05-20 §
21:07 <ejegg> updated payments-wiki from 8397ccf9cc to d5ef5ad067 [production]
19:19 <mobrovac> bootstrap restbase1027-a - T219404 [production]
18:55 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.5/includes/Linker.php: T222857 / Iecc2140fabd3 (duration: 00m 54s) [production]
16:43 <onimisionipe> rolling reboot of maps eqiad to pick kernel upgrades [production]
16:38 <mobrovac> bootstrap restbase1026-c - T219404 [production]
15:26 <onimisionipe> rebooting codfw maps to pick up kernel upgrades [production]
15:26 <marostegui> Stop replication on labsdb1011 to start compressing tables - T222978 [production]