|
2018-03-12
§
|
| 09:56 |
<elukey> |
restart kafka mirror maker (main eqiad -> jumbo) on kafka1020 (all consumers not assigned to any partition on kafka102*) |
[production] |
| 09:53 |
<moritzm> |
installing util-linux security updates |
[production] |
| 09:31 |
<_joe_> |
decommission mw2097-mw2134 from conftool T189111 |
[production] |
| 08:40 |
<moritzm> |
rebooting iron for kernel security update |
[production] |
| 08:32 |
<ema> |
cp3033/cp3031: restart varnish-be |
[production] |
| 08:24 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool es2015 after kernel upgrade (duration: 00m 58s) |
[production] |
| 08:20 |
<ema> |
cp3033/cp3031: set transaction_timeout to 60s |
[production] |
| 08:14 |
<marostegui> |
Stop MySQL on es2015 for kernel upgrade |
[production] |
| 08:06 |
<ema> |
cp3042: restart varnish-be |
[production] |
| 08:03 |
<ema> |
cp3042: set transaction_timeout to 30s |
[production] |
| 07:45 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool es2015 for kernel upgrade (duration: 00m 58s) |
[production] |
| 07:38 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool es2014 after kernel upgrade (duration: 01m 01s) |
[production] |
| 07:35 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool es2014 for kernel upgrade (duration: 00m 59s) |
[production] |
| 07:26 |
<marostegui> |
Stop MySQL on es2014 for kernel upgrade |
[production] |
| 07:24 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool es2014 for kernel upgrade (duration: 00m 58s) |
[production] |
| 07:10 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1113:3316 as vslow,dump in s6 - T184161 (duration: 00m 58s) |
[production] |
| 06:54 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1113:3315 as vslow,dump in s5 - T184161 (duration: 00m 58s) |
[production] |
| 06:27 |
<marostegui> |
Deploy schema change on db1103:3314 - T187089 T185128 T153182 |
[production] |
| 06:25 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1103:3314 for alter table (duration: 01m 06s) |
[production] |
| 02:52 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.24) (duration: 11m 56s) |
[production] |
|
2018-03-09
§
|
| 23:29 |
<tgr@tin> |
Synchronized php-1.31.0-wmf.24/extensions/ReadingLists/src/Api/ApiQueryReadingListEntries.php: T189272 fix stupid ReadingLists typo breaking production (duration: 00m 54s) |
[production] |
| 19:43 |
<foks> |
changed global email for User:Mathmensch |
[production] |
| 19:19 |
<MaxSem> |
restarted my script on tin, now with more aggressive writes |
[production] |
| 18:26 |
<reedy@tin> |
Synchronized php-1.31.0-wmf.24/extensions/AbuseFilter/includes/AbuseFilter.class.php: Unbreak AbuseFilter tagging T189299 (duration: 00m 59s) |
[production] |
| 17:35 |
<andrew@tin> |
Finished deploy [horizon/deploy@9c234d6]: Another try at fixing T188458 (duration: 03m 00s) |
[production] |
| 17:32 |
<andrew@tin> |
Started deploy [horizon/deploy@9c234d6]: Another try at fixing T188458 |
[production] |
| 16:14 |
<andrewbogott> |
test log |
[production] |
| 16:07 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp3034.esams.wmnet |
[production] |
| 16:07 |
<andrewbogott> |
what about now, can you log now? |
[production] |
| 16:03 |
<andrewbogott> |
test log to new labweb/wikitech cluster |
[production] |
| 15:59 |
<ppchelko@tin> |
Finished deploy [cpjobqueue/deploy@5795526]: Enable root_claim_ttl for refreshLinks T189303 (duration: 00m 38s) |
[production] |
| 15:59 |
<andrewbogott> |
moving wikitech dns record to point to misc-web and the new labweb cluster, https://gerrit.wikimedia.org/r/#/c/417926/ |
[production] |
| 15:59 |
<ppchelko@tin> |
Started deploy [cpjobqueue/deploy@5795526]: Enable root_claim_ttl for refreshLinks T189303 |
[production] |
| 15:54 |
<andrew@tin> |
Finished deploy [horizon/deploy@f59f568]: rolling out a fix for T188458 (duration: 03m 11s) |
[production] |
| 15:51 |
<andrew@tin> |
Started deploy [horizon/deploy@f59f568]: rolling out a fix for T188458 |
[production] |
| 15:30 |
<moritzm> |
installing zsh security update on trusty |
[production] |
| 15:20 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1063 after cloning db1113:3316 - T184161 (duration: 00m 58s) |
[production] |
| 15:15 |
<moritzm> |
installing sensible-utils security update on trusty (Debian already fixed) |
[production] |
| 15:11 |
<ema> |
cp-upload_esams: reboot for retpoline kernel updates T188092 |
[production] |
| 13:12 |
<marostegui> |
Compress s6 on db1113:3316 - T184161 |
[production] |
| 12:41 |
<elukey> |
manually executed systemctl reset-failed to some old (not present anymore) units on kafka analytics hosts |
[production] |
| 12:26 |
<marostegui> |
Compress s5 on db1113:3315 - T184161 |
[production] |
| 12:16 |
<marostegui> |
Stop mysql on db1063 to clone db1113:3316 - T184161 |
[production] |
| 12:16 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1063 to clone db1113:3316 - T184161 (duration: 00m 58s) |
[production] |
| 12:11 |
<jynus> |
dropping test databases on dbstore2* instances |
[production] |
| 12:08 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1051 after cloning db1113:3315- T184161 (duration: 00m 58s) |
[production] |
| 12:06 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db1051 after cloning db1113:3315- T184161 (duration: 00m 58s) |
[production] |