|
2017-06-19
§
|
| 10:15 |
<godog> |
roll-upgrade swift to 2.10 on to ms-fe1* - T162609 |
[production] |
| 09:56 |
<akosiaris> |
migrate neon.eqiad.wmnet to ganeti01.svc.eqiad.wmnet's row_A nodegroup |
[production] |
| 09:55 |
<dcausse> |
restarting elasticsearch on relforge1* to pickup new snapshot of the ltr plugin |
[production] |
| 09:33 |
<jynus> |
temporarily stop dbstore1002:s3 and db1015 to fix srwiki |
[production] |
| 09:30 |
<marostegui> |
Deploy alter table on s2 - dbstore1001 - T166205 |
[production] |
| 09:18 |
<godog> |
swift eqiad-prod: remove ms-be1001 - ms-be1012 - T166489 |
[production] |
| 09:13 |
<paravoid> |
rebooting achernar to address CPU throttling and apply the BIOS update |
[production] |
| 09:11 |
<paravoid> |
upgrading achernar's BIOS from 1.2.4 to 2.4.2 hoping it will address recurring CPU throttling issue (T162850) |
[production] |
| 09:07 |
<akosiaris> |
restart ircecho on einsteinium, was not notifying due to a thrown exception |
[production] |
| 08:35 |
<marostegui> |
Drop table title key from s2 - T164949 |
[production] |
| 08:16 |
<marostegui> |
Drop table titlekey on s6 - T164949 |
[production] |
| 07:59 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool pc2004,5,6 after maintenance (duration: 00m 41s) |
[production] |
| 07:42 |
<moritzm> |
restarting app server canaries to pick up gnutls update |
[production] |
| 07:13 |
<marostegui> |
Reboot ms-be1010 |
[production] |
| 07:10 |
<marostegui> |
Deploy alter table s5 - codfw master - db2023 (and will replicate) so this will generate lag on codfw slaves - T166207 |
[production] |
| 07:09 |
<jynus> |
upgrade, reboot and clear data on pc2006 |
[production] |
| 07:05 |
<jynus> |
upgrade, reboot and clear data on pc2005 |
[production] |
| 07:03 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Depool pc2005 & pc2006 (duration: 00m 41s) |
[production] |
| 06:58 |
<moritzm> |
installing gnutls security updates |
[production] |
| 06:38 |
<marostegui> |
Deploy alter table s2 - labsdb1001 - T166205 |
[production] |
| 06:37 |
<jynus> |
force learning cycle to db1046 controller T166141 |
[production] |
| 06:23 |
<marostegui> |
Deploy alter table on s2 - db1021 - T166205 |
[production] |
| 06:18 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1021 - T166205 (duration: 00m 41s) |
[production] |
| 04:21 |
<reedy@tin> |
Synchronized composer.lock: update (duration: 00m 41s) |
[production] |
| 04:20 |
<reedy@tin> |
Synchronized composer.json: update (duration: 00m 41s) |
[production] |
| 04:19 |
<reedy@tin> |
Synchronized multiversion/vendor/: Update! (duration: 01m 05s) |
[production] |
| 04:05 |
<reedy@tin> |
Synchronized wmf-config/CommonSettings.php: Fix comments minor code style (duration: 00m 42s) |
[production] |
| 02:26 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Jun 19 02:26:06 UTC 2017 (duration 6m 8s) |
[production] |
| 02:19 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 04s) |
[production] |
|
2017-06-16
§
|
| 19:54 |
<Reedy> |
disabled cluster 2fa for Chrissymad for T168064 (confirmed by email) |
[production] |
| 19:26 |
<ejegg> |
re-enabled paypal audit download and parse job |
[production] |
| 19:13 |
<ebernhardson> |
restarting elasticesarch on relforge to pick up new ltr-query plugin version |
[production] |
| 18:14 |
<mutante> |
ms-be1001: did not change config, tried again, now detected 13 drives again, coming back |
[production] |
| 18:10 |
<mutante> |
ms-be1001 - The following VDs are missing: 09 |
[production] |
| 18:08 |
<mutante> |
ms-be1001 - powercycling crashed server - "[14076481.245487] general protection fault: 0000 [#4] SMP |
[production] |
| 13:36 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Remove comments from db1018 current status - T166205 (duration: 00m 41s) |
[production] |
| 13:26 |
<twentyafterfour> |
fixed phabricator "upgrade database" error. |
[production] |
| 13:20 |
<twentyafterfour> |
fixing phab database migrations |
[production] |
| 13:01 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1091 after performance testing (duration: 00m 41s) |
[production] |
| 10:18 |
<jynus> |
running analyze on db1091 (depooled), may create lag |
[production] |
| 10:11 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1091 for performance testing (duration: 00m 42s) |
[production] |
| 09:52 |
<moritzm> |
installing guile security updates |
[production] |
| 09:13 |
<moritzm> |
re-enabled puppet on mw2129 (no reason was given why it was disabled( |
[production] |