|
2018-08-16
§
|
| 10:30 |
<_joe_> |
restarting changeprop on scb1002, not listening on its tcp port |
[production] |
| 10:27 |
<_joe_> |
restarting cpjobqueue on scb1002, not listening on its tcp port |
[production] |
| 10:00 |
<volans> |
add jiji to the 'ops' LDAP group - T201849 |
[production] |
| 09:46 |
<gehel> |
all elasticsearch nodes reimaged (except elastic1029, waiting on memory issue) - T198391 / T193649 / T201991 |
[production] |
| 09:16 |
<gehel> |
reimaging elastic1022 |
[production] |
| 08:40 |
<moritzm> |
upgrading wikidiff to 1.7.2 on mw1319-mw1333 (HHVM bytecode cache is pruned during update) |
[production] |
| 08:37 |
<ema> |
reboot cp2009 for kernel upgrade |
[production] |
| 08:35 |
<vgutierrez> |
Reimaging radon as spare system - T202040 |
[production] |
| 08:32 |
<moritzm> |
uploaded jenkins 2.121.3 to apt.wikimedia.org (for jessie and stretch) |
[production] |
| 08:16 |
<moritzm> |
upgrading wikidiff to 1.7.2 on mw1334-mw1338/mw1307/mw1318/ (HHVM bytecode cache is pruned during update) |
[production] |
| 07:49 |
<gehel> |
reimaging elastic10(23|24) |
[production] |
| 07:45 |
<volans@deploy1001> |
Finished deploy [debmonitor/deploy@1f01fd1]: Release v0.1.8 (duration: 00m 31s) |
[production] |
| 07:45 |
<volans@deploy1001> |
Started deploy [debmonitor/deploy@1f01fd1]: Release v0.1.8 |
[production] |
| 07:27 |
<moritzm> |
rebooting install1002 for kernel security update |
[production] |
| 07:24 |
<moritzm> |
rebooting install2002 for kernel security update |
[production] |
| 05:12 |
<_joe_> |
moving away corrupted commitlog file on aqs1007 cassandra-a instance, trying to restart it |
[production] |
| 02:38 |
<l10nupdate@deploy1001> |
ResourceLoader cache refresh completed at Thu Aug 16 02:38:46 UTC 2018 (duration 10m 17s) |
[production] |
| 02:28 |
<l10nupdate@deploy1001> |
scap sync-l10n completed (1.32.0-wmf.16) (duration: 08m 38s) |
[production] |
|
2018-08-15
§
|
| 22:43 |
<krinkle@deploy1001> |
Finished scap: rm StartProfiler.php - T201782 (duration: 31m 46s) |
[production] |
| 22:33 |
<XioNoX> |
update old eqiad recdns IPs in cr1/2-eqiad bgp group Anycast4 with dns1001/2 |
[production] |
| 22:11 |
<krinkle@deploy1001> |
Started scap: rm StartProfiler.php - T201782 |
[production] |
| 21:57 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.10: rm StartProfiler.php - T201782 (duration: 03m 01s) |
[production] |
| 21:48 |
<krinkle@deploy1001> |
Synchronized wmf-config/: I173a02910c (duration: 00m 50s) |
[production] |
| 21:46 |
<krinkle@deploy1001> |
Synchronized scap/plugins/: I173a02910c (duration: 00m 50s) |
[production] |
| 20:49 |
<krinkle@deploy1001> |
Synchronized wmf-config/CommonSettings.php: I207421c87eb6d2f1 (duration: 00m 57s) |
[production] |
| 19:07 |
<gehel> |
reimaging elastic10(18|19|20) |
[production] |
| 17:20 |
<XioNoX> |
Rollback: redirect ns2 to authdns1001 - T201608 |
[production] |
| 17:16 |
<moritzm> |
reboot eeden for kernel security update |
[production] |
| 17:07 |
<XioNoX> |
redirect ns2 to authdns1001 - T201608 |
[production] |
| 17:00 |
<moritzm> |
reboot radon for kernel security update |
[production] |
| 16:50 |
<XioNoX> |
redirect ns0 to authdns1001 - T201608 |
[production] |
| 16:42 |
<XioNoX> |
rollback: redirect ns1 to radon - T201608 |
[production] |
| 16:35 |
<moritzm> |
reboot authdns2001 for kernel security update |
[production] |
| 16:27 |
<XioNoX> |
redirect ns1 to radon - T201608 |
[production] |
| 14:58 |
<gehel> |
reimaging elastic1028 |
[production] |
| 14:50 |
<ejegg> |
disabled CiviCRM dedupe jobs |
[production] |
| 14:47 |
<ejegg> |
updated SmashPig free-standing deployment from 2736e41045 to 3603e191a3 |
[production] |
| 14:13 |
<gehel> |
reimaging elastic102[567] |
[production] |
| 13:47 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=mw2184.codfw.wmnet |
[production] |
| 13:35 |
<moritzm> |
installing samba security updates |
[production] |
| 12:51 |
<Trey314159> |
reindexing Polish wikis on elastic@eqiad and elastic@codfw complete (T200037) |
[production] |
| 12:22 |
<moritzm> |
rebooting mw2190-mw2219 for kernel security update (also bundling wikidiff and apache updates) |
[production] |
| 12:19 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=mw2184.codfw.wmnet |
[production] |
| 12:04 |
<ladsgroup@deploy1001> |
Synchronized php-1.32.0-wmf.16/includes/changetags/ChangeTags.php: [[gerrit:452921|Swap SET and WHERE statements in ChangeTags::undefineTag (T201934)]] (duration: 00m 55s) |
[production] |
| 10:52 |
<moritzm> |
rebooting mw2163-mw2189 for kernel security update (also bundling wikidiff and apache updates) |
[production] |
| 09:39 |
<gehel> |
reimaging elastic103[012], this will trigger master re-election |
[production] |
| 09:17 |
<moritzm> |
rebooting mw2135-mw2147 for kernel security update (also bundling wikidiff and apache updates) |
[production] |
| 08:59 |
<moritzm> |
rebooting deployment-mediawiki-09 |
[production] |
| 08:14 |
<gehel> |
masking cassandra-a instance on aqs1007 since it is flapping - T201986 |
[production] |
| 02:45 |
<l10nupdate@deploy1001> |
ResourceLoader cache refresh completed at Wed Aug 15 02:45:47 UTC 2018 (duration 10m 24s) |
[production] |