8801-8850 of 10000 results (63ms)
2019-05-21 §
08:56 <marostegui> Stop MySQL on db2041 as it will be decommissioned T223950 [production]
08:24 <hashar> Updated phan jobs to no more install mediawiki development dependencies (potentially conflicting with the extensions code) https://gerrit.wikimedia.org/r/#/c/integration/config/+/511447/ # T223397 [releng]
07:29 <legoktm> deployed https://gerrit.wikimedia.org/r/511627 [releng]
07:10 <legoktm> manually rebuilding mediawiki-core-code-coverage-docker with 5 hour timeout [releng]
06:59 <oblivian@deploy1001> Synchronized wmf-config/CommonSettings.php: Turning off php7 sampling for investigation in T223952 (duration: 00m 53s) [production]
06:55 <elukey> reboot of stat100[4,5,6,7] and notebook100[3,4] for kernel upgrades [production]
06:31 <marostegui> Stop mariadb on db2104 to convert it to s2 candidate master [production]
06:30 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2104 (duration: 00m 51s) [production]
06:28 <elukey> chown analytics:analytics /user/hdfs/salts/eventlogging_sanitization on HDFS [analytics]
05:50 <marostegui> Remove db2041 from tendril and zarcillo - T223950 [production]
05:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db2041 for decommissioning T223950 (duration: 00m 51s) [production]
05:42 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db2041 for decommissioning T223950 (duration: 00m 51s) [production]
05:16 <marostegui> Stop MySQL on db2040 [production]
05:16 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2040 (duration: 00m 50s) [production]
05:14 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Pool db2114 into s6 - T222772 (duration: 00m 50s) [production]
05:13 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Pool db2114 into s6 - T222772 (duration: 00m 51s) [production]
03:36 <urandom> bootstrapping restbase1027-c -- T219404 [production]
00:47 <urandom> bootstrapping restbase1027-b -- T219404 [production]
00:05 <aaron@deploy1001> Synchronized php-1.34.0-wmf.5/includes/libs/objectcache/APCUBagOStuff.php: 982299d635623279 (duration: 00m 54s) [production]
2019-05-20 §
21:07 <ejegg> updated payments-wiki from 8397ccf9cc to d5ef5ad067 [production]
19:19 <mobrovac> bootstrap restbase1027-a - T219404 [production]
19:01 <wm-bot> <jeanfred> Restart uwsgi with 10 workers rather than the default of 4 [tools.integraality]
19:00 <wm-bot> <jeanfred> --help [tools.integraality]
18:55 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.5/includes/Linker.php: T222857 / Iecc2140fabd3 (duration: 00m 54s) [production]
17:25 <arturo> T223923 dropped compat-network config from /etc/network/interfaces in eqiad1/codfw1dev neutron nodes [admin]
17:22 <arturo> T223923 dropped br-compat bridges and vlan interfaces (1102 and 2102) in eqiad1/codfw1dev neutron nodes [admin]
17:17 <elukey> chown -R analytics:analytics /tmp/DataFrameToDruid on HDFS [analytics]
17:07 <arturo> T223923 dropped compat-network configuration from the neutron database in eqiad1 [admin]
16:55 <arturo> T223923 dropped compat-network configuration from the neutron database in codfw1dev [admin]
16:43 <onimisionipe> rolling reboot of maps eqiad to pick kernel upgrades [production]
16:39 <joal> Manually run webrequest-load-wf-upload-2019-5-20-11 with higher error threshold as error were false positive [analytics]
16:38 <mobrovac> bootstrap restbase1026-c - T219404 [production]
15:28 <joal> Rerunning timeout webrequest-load-coord-text and webrequest-load-coord-upload (2019-05-20T09:00) [analytics]
15:26 <onimisionipe> rebooting codfw maps to pick up kernel upgrades [production]
15:26 <marostegui> Stop replication on labsdb1011 to start compressing tables - T222978 [production]
15:13 <anomie@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Setting actor migration to write-new/read-new on group 0 (T188327) (duration: 00m 55s) [production]
14:54 <bblack> rebooting lvs1013, lvs1014, lvs1015 (not in active service, yet) [production]
14:43 <jiji@deploy1001> Finished deploy [cpjobqueue/deploy@89b0ad0]: Migrating RecordLintJob to PHP7 - T219148 (duration: 00m 55s) [production]
14:42 <jiji@deploy1001> Started deploy [cpjobqueue/deploy@89b0ad0]: Migrating RecordLintJob to PHP7 - T219148 [production]
14:41 <elukey> chown analytics:analytics /wmf/data/event_sanitized on HDFS [analytics]
14:21 <marostegui> Reload haproxy on dbroxy1010 to depool labsdb1011 [production]
14:14 <marostegui> Reload haproxy on dbroxy1010 to repool labsdb1010 [production]
13:58 <mobrovac> bootstrap restbase1026-b - T219404 [production]
13:11 <hashar> updating phan jobs to use docker-registry.wikimedia.org/releng/mediawiki-phan:0.1.15 # T219114 [releng]
12:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1126 and db1134 (duration: 00m 50s) [production]
12:02 <elukey> chown analytics:analytics /wmf/data/event on HDFS [analytics]
12:00 <elukey> chown analytics:analytics /wmf/data/wmf/event on HDFS [analytics]
11:44 <fsero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:44 <fsero@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:28 <fsero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]