|
2018-12-19
§
|
| 09:14 |
<moritzm> |
rebooting restbase2008 for kernel security update |
[production] |
| 08:53 |
<elukey> |
roll restart of cassandra on aqs1005-1009 for opendjdk upgrades |
[production] |
| 08:50 |
<marostegui> |
Drop image_comment_temp from s7 - T209591 |
[production] |
| 08:46 |
<marostegui> |
Drop image_comment_temp from s6 - T209591 |
[production] |
| 08:43 |
<akosiaris> |
rebalance row_A ganeti01.svc.codfw.wmnet nodegroup after recabling T210447 |
[production] |
| 08:40 |
<marostegui> |
Drop image_comment_temp from s8 - T209591 |
[production] |
| 08:37 |
<moritzm> |
draining restbase2007 for eventual reboot for kernel security update |
[production] |
| 08:28 |
<marostegui> |
Drop image_comment_temp from s2 - T209591 |
[production] |
| 08:27 |
<marostegui> |
Drop image_comment_temp from s1 - T209591 |
[production] |
| 08:16 |
<godog> |
swift eqiad-prod: more weight for ms-be10[44-50].eqiad.wmnet - T209618 |
[production] |
| 07:37 |
<marostegui> |
Drop nodepooldb on m5 master - T212230 |
[production] |
| 07:22 |
<marostegui> |
Stop MySQL on db2050 to clone db2057 - T212275 |
[production] |
| 07:15 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2050 to clone db2057 T212275 (duration: 00m 52s) |
[production] |
| 06:59 |
<marostegui> |
Enable GTID on s8 codfw master (db2045) - T211973 |
[production] |
| 06:58 |
<marostegui> |
Enable GTID on s1 codfw master (db2048) - T211973 |
[production] |
| 06:44 |
<marostegui> |
Remove nodepool@10.64.16.155 user from m5 master - T212230 |
[production] |
| 06:34 |
<marostegui> |
Hard reboot db2057 - T212275 |
[production] |
| 06:27 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2057 - storage crashed T212275 (duration: 01m 08s) |
[production] |
| 01:27 |
<tstarling@deploy1001> |
Synchronized php-1.33.0-wmf.8/extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php: g 480681 make maintenance script dry run more useful (duration: 00m 52s) |
[production] |
| 01:25 |
<tstarling@deploy1001> |
Synchronized php-1.33.0-wmf.8/extensions/AbuseFilter/includes/AbuseFilter.php: g 480680 fix exception in maintenance script (duration: 00m 54s) |
[production] |
| 00:11 |
<mutante> |
contint1001 - rsyncing /srv/org/wikimedia/docs to rsync://docs1001.eqiad.wmnet/docs T211974 |
[production] |
|
2018-12-18
§
|
| 23:49 |
<XioNoX> |
remove BGP session to AS50629 from cr2-esams (not in AMS-IX anymore) |
[production] |
| 20:33 |
<otto@deploy1001> |
Finished deploy [eventlogging/analytics@104adb5]: Send JSON string of event for validation errors in EventError (duration: 00m 04s) |
[production] |
| 20:33 |
<otto@deploy1001> |
Started deploy [eventlogging/analytics@104adb5]: Send JSON string of event for validation errors in EventError |
[production] |
| 19:43 |
<mholloway-shell@deploy1001> |
Finished deploy [mobileapps/deploy@963d704]: Parse summaries from lead objects only (T202642) (duration: 05m 26s) |
[production] |
| 19:38 |
<mholloway-shell@deploy1001> |
Started deploy [mobileapps/deploy@963d704]: Parse summaries from lead objects only (T202642) |
[production] |
| 19:30 |
<robh> |
migration of ulsfo pdus complete |
[production] |
| 19:19 |
<sbisson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:480106|Production configuration for GrowthExperiments Help Panel]] (duration: 00m 52s) |
[production] |
| 19:06 |
<XioNoX> |
redirect ns1 back to authdns2001 - T210447 |
[production] |
| 18:51 |
<moritzm> |
installing libx11 security updates |
[production] |
| 18:48 |
<XioNoX> |
depool ulsfo - T209101 |
[production] |
| 18:42 |
<XioNoX> |
repool codfw - T210447 |
[production] |
| 18:40 |
<XioNoX> |
Revert "Redirect eqsin/ulsfo caches to eqiad" - T210447 |
[production] |
| 18:05 |
<chasemp> |
stat1004:~# umount /mnt/T211327 |
[production] |
| 17:58 |
<XioNoX> |
shutdown fpc4 for replacement - T210447 |
[production] |
| 17:51 |
<bblack> |
deploying php7 cache-splitter patch to cache_text - https://gerrit.wikimedia.org/r/c/operations/puppet/+/478680 - T206339 |
[production] |
| 17:49 |
<godog> |
bounce rsyslog on lithium, tls listener timeout |
[production] |
| 17:34 |
<volans> |
triggered restart of ircecho on icinga1001 while applying https://gerrit.wikimedia.org/r/480509 |
[production] |
| 16:37 |
<jijiki> |
librsvg* 2.40.20-3+wmf1+stretch1 uploaded to components/thumor to stretch-wikimedia - T209886 |
[production] |
| 16:22 |
<bblack> |
powercycle cp1075 from console (crashed, apparently) |
[production] |
| 16:07 |
<XioNoX> |
starting codfw row A recabling - T210447 |
[production] |
| 16:05 |
<ejegg> |
updated fundraising python tools from af5dbee8eb to 5f44d9dd43 |
[production] |
| 15:53 |
<XioNoX> |
redirect ns1 to authdns1001 for T210447 |
[production] |
| 15:47 |
<XioNoX> |
redirect eqsin/ulsfo caches to eqiad for T210447 |
[production] |
| 15:44 |
<XioNoX> |
depool codfw for T210447 |
[production] |
| 15:30 |
<akosiaris> |
empty ganeti2005, ganeti2006 for T210447 |
[production] |
| 15:16 |
<Amir1> |
mwscript extensions/WikibaseQualityConstraints/maintenance/ImportConstraintEntities.php --wiki=testwikidatawiki --config-format=wgConf | tee WikibaseQualityConstraints-config.php |
[production] |
| 14:55 |
<akosiaris> |
restart pybal on lvs1006, lvs2003 for blubberoid LVS deployment. T205919 |
[production] |
| 14:40 |
<zfilipin@deploy1001> |
rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.9 |
[production] |
| 14:26 |
<akosiaris> |
restart ircecho, seems to have croaked with Dec 17 19:39:52 icinga1001 ircecho[861]: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 174: ordinal not in range(128) |
[production] |