|
2016-11-08
ยง
|
| 13:42 |
<apergos> |
deferring reboots of mw1204-1216 and mw1170-1188 for a while |
[production] |
| 13:06 |
<moritzm> |
rolling reboot of restbase-test for kernel update |
[production] |
| 12:52 |
<ema> |
upgrading pinkunicorn to varnish 4.1.3-1wm3 T150247 |
[production] |
| 12:41 |
<moritzm> |
restarted ntp on mw1194, stuck in XFAC state |
[production] |
| 12:39 |
<moritzm> |
rebooting iron for kernel update |
[production] |
| 12:28 |
<moritzm> |
restarted ntp on mw1166, stuck in XFAC state |
[production] |
| 12:24 |
<moritzm> |
depooling/rebooting/repooling scb1002 for kernel update |
[production] |
| 12:23 |
<moritzm> |
restarted ntp on hafnium, stuck in XFAC state |
[production] |
| 12:12 |
<moritzm> |
depooling/rebooting/repooling scb1001 for kernel update |
[production] |
| 12:11 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Pool back db1051 and api servers to high load after hw issues (duration: 02m 45s) |
[production] |
| 11:59 |
<apergos> |
rolling restart of mw1170-1216 for new kernel |
[production] |
| 11:54 |
<apergos> |
restart of mw1240, 1253 for new kernel |
[production] |
| 11:44 |
<moritzm> |
rebooting logstash1001 for kernel update |
[production] |
| 11:42 |
<moritzm> |
rearmed keyholder on mira |
[production] |
| 11:38 |
<apergos> |
rolling reboot of mw1161, mw1163-1169 for new kernel |
[production] |
| 11:37 |
<jynus> |
running schema change on db1045 (pagelinks) T139090 |
[production] |
| 11:30 |
<moritzm> |
rebooting mira for kernel update |
[production] |
| 11:19 |
<apergos> |
rolling restart of mw2080-2085 for new kernel |
[production] |
| 11:15 |
<jynus> |
running schema change on db2070 (pagelinks) T139090 |
[production] |
| 11:02 |
<mark> |
Activated cr2-eqiad bgp group IX4 |
[production] |
| 10:53 |
<apergos> |
rolling reboot of mw2090 - mw2096 for new kernel |
[production] |
| 10:41 |
<moritzm> |
rearmed keyholder on tin |
[production] |
| 10:39 |
<apergos> |
rebooting mw2086 - mw2089 for new kernel |
[production] |
| 10:36 |
<jynus> |
rebooting and upgrading db2012 |
[production] |
| 10:33 |
<moritzm> |
rebooting tin for kernel update |
[production] |
| 10:23 |
<moritzm> |
restarted ntp on mw2075, stuck in XFAC state |
[production] |
| 10:22 |
<moritzm> |
rebooting hafnium for kernel update |
[production] |
| 10:08 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1059 - T149079 T147305 (duration: 00m 57s) |
[production] |
| 10:06 |
<moritzm> |
rebooting rhenium for kernel update |
[production] |
| 09:59 |
<moritzm> |
rebooting oxygen for kernel update |
[production] |
| 09:57 |
<apergos> |
rebooting mw2075 - mw2079 for new kernel |
[production] |
| 09:49 |
<moritzm> |
rebooting install2001 for kernel update |
[production] |
| 09:42 |
<moritzm> |
rebooting bast2001 for kernel update |
[production] |
| 09:24 |
<moritzm> |
rebooting graphite1002 for kernel update |
[production] |
| 08:44 |
<marostegui> |
Deploy schema change s4 commonswiki.revision table - T147305 |
[production] |
| 08:21 |
<moritzm> |
rolling reboot of swift backend servers in esams for kernel update |
[production] |
| 08:09 |
<moritzm> |
rolling reboot of parsoid in eqiad for kernel update |
[production] |
| 08:04 |
<elukey> |
rebooting stat1001 for kernel upgrades (will cause a brief unavail for analytics websites) |
[production] |
| 07:30 |
<marostegui> |
Deploy schema change s5 dewiki.revision on codfw master (db2023) - T148967 |
[production] |
| 07:10 |
<_joe_> |
stopped logstash, removed large logfiles that were erroneously non-rotated, started logstash across the logstash cluster |
[production] |
| 02:31 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Nov 8 02:31:30 UTC 2016 (duration 4m 16s) |
[production] |
| 02:27 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.1) (duration: 09m 52s) |
[production] |
| 01:48 |
<eileen1> |
update CiviCRM from 45a1b9a0e1665b1fc71165e1bc4bcfdca2a9adf7 to 63d35fc5f5c065168efd2250c2741b3603b2eb92 |
[production] |
| 01:27 |
<eileen1> |
enabled GlobalCollect Recurring Donations Dedupe CiviCRM Contacts GlobalCollect audit file download Fundraiser Public Data Export |
[production] |
| 01:23 |
<eileen1> |
enabled Thank you mail send |
[production] |
| 01:17 |
<eileen1> |
disabled Public Data Export |
[production] |
| 01:06 |
<eileen1> |
disable GlobalCollect audit file |
[production] |
| 01:05 |
<eileen1> |
disable jobs http://localhost:9000/job/Dedupe%20CiviCRM%20contacts/ http://localhost:9000/job/Dedupe%20CiviCRM%20contacts%20(name-match)/ Thank you mail send |
[production] |
| 01:04 |
<eileen1> |
disable jobs GlobalCollect Recurring Donations |
[production] |
| 00:50 |
<godog> |
swift eqiad-prod: set weight for ms-be1021 sd[h-n] to 3000 - T139767 |
[production] |