|
2018-05-17
§
|
| 08:04 |
<jynus> |
stop and reimage db2056 |
[production] |
| 07:53 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1093 with low load (duration: 01m 20s) |
[production] |
| 07:25 |
<elukey> |
bounced all the prometheus burrow exporters on kafkamon* hosts to refresh their metrics and drop old/expired cgroups |
[production] |
| 07:22 |
<marostegui> |
Deploy schema change on db1090:3317 - T191519 T188299 T190148 |
[production] |
| 07:22 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1090:3317 for alter table (duration: 01m 21s) |
[production] |
| 07:19 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Specify the sanitarium masters in codfw (duration: 01m 21s) |
[production] |
| 07:12 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1079 after alter table (duration: 01m 20s) |
[production] |
| 07:05 |
<jynus> |
stop and reimage db1093 |
[production] |
| 07:02 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1093 (duration: 01m 22s) |
[production] |
| 05:52 |
<marostegui> |
Disable BBU auto-learn on new hosts - T192979 |
[production] |
| 05:31 |
<marostegui> |
Deploy schema change on db1079 with replication (this will generate lag on labs s7) - T191519 T188299 T190148 |
[production] |
| 05:31 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1079 for alter table (duration: 01m 22s) |
[production] |
| 05:20 |
<marostegui> |
Force BBU learn cycle on db1054 - T194867 |
[production] |
| 05:18 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1098:3317 after alter table (duration: 01m 48s) |
[production] |
| 05:09 |
<marostegui> |
Deploy schema change on s4 primary master (db1068) - T191519 T188299 T190148 |
[production] |
| 04:10 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu May 17 04:10:13 UTC 2018 (duration 7m 42s) |
[production] |
| 04:02 |
<l10nupdate@tin> |
scap sync-l10n completed (1.32.0-wmf.4) (duration: 14m 08s) |
[production] |
| 03:04 |
<l10nupdate@tin> |
scap sync-l10n completed (1.32.0-wmf.3) (duration: 14m 27s) |
[production] |
| 00:57 |
<mutante> |
installing OS on webperf1002, webperf2002 |
[production] |
|
2018-05-16
§
|
| 21:58 |
<cmjohnson1> |
swapping PEM 1 asw-c8-eqiad |
[production] |
| 20:13 |
<marostegui> |
Force WriteBack policy on db1067 - T194852 |
[production] |
| 19:38 |
<twentyafterfour> |
The train for 1.32.0-wmf.4 is blocked by fatals in Echo extension. See T194848 |
[production] |
| 19:28 |
<twentyafterfour@tin> |
Synchronized php-1.32.0-wmf.4/extensions/CongressLookup: sync CongressLookup extension on wmf.4 refs T191050 (duration: 01m 22s) |
[production] |
| 17:35 |
<milimetric@tin> |
Finished deploy [analytics/refinery@a205447]: Fix drop partitions script (duration: 15m 12s) |
[production] |
| 17:33 |
<hoo@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable RemexHtml on wikis with < 100 ns0 errors in high priority cats (T193685) (duration: 01m 22s) |
[production] |
| 17:20 |
<milimetric@tin> |
Started deploy [analytics/refinery@a205447]: Fix drop partitions script |
[production] |
| 17:16 |
<milimetric@tin> |
Started deploy [analytics/refinery@15be6ae]: Fix drop partitions script |
[production] |
| 16:38 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1088 with low weight (duration: 01m 21s) |
[production] |
| 16:34 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Change db1067 IP - T193835 (duration: 01m 21s) |
[production] |
| 16:29 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Change db1067 IP - T193835 (duration: 01m 17s) |
[production] |
| 16:22 |
<elukey> |
upgrade burrow on kafkamon1001 from 1.0 to 1.1 |
[production] |
| 16:18 |
<marostegui> |
Power off db1067 for rack move - T193835 |
[production] |
| 16:18 |
<godog> |
move ms-be1036 for asw2-c-eqiad - T187962 |
[production] |
| 16:03 |
<godog> |
move ms-be1035 for asw2-c-eqiad - T187962 |
[production] |
| 16:03 |
<_joe_> |
installing cergen 0.2.2 on the puppetmaster frontends |
[production] |
| 15:50 |
<andrewbogott> |
upgrading kernel, microcode and rebooting labnet1002 |
[production] |
| 15:50 |
<jynus> |
stop and reimage db1088 |
[production] |
| 15:49 |
<godog> |
move ms-be1034 for asw2-c-eqiad - T187962 |
[production] |
| 15:36 |
<godog> |
move ms-be1025 for asw2-c-eqiad - T187962 |
[production] |
| 15:35 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1088 (duration: 03m 26s) |
[production] |
| 15:27 |
<_joe_> |
uploading cergen 0.2.2 to stretch, jessie |
[production] |
| 15:18 |
<godog> |
move ms-be1024 for asw2-c-eqiad - T187962 |
[production] |
| 15:17 |
<elukey> |
upgrade burrow from 1.0.0 to 1.1.0 on kafkamon* hosts |
[production] |
| 15:17 |
<elukey> |
upload burrow 1.1.0 to stretch|jessie-wikimedia |
[production] |
| 15:17 |
<bstorm_> |
labsdb1004 reboot and upgrade kernel to 4.9.0-0.bpo.6-amd64 |
[production] |
| 15:10 |
<godog> |
pool ms-fe1008 for asw2 move - T187962 |
[production] |
| 15:02 |
<marostegui> |
Stop MySQL on labsdb1004 |
[production] |
| 15:01 |
<marostegui> |
Stop MySQL on db1067 - T193835 |
[production] |
| 14:54 |
<godog> |
depool ms-fe1008 for asw2 move - T187962 |
[production] |
| 14:50 |
<moritzm> |
upgrading codfw job runners to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 |
[production] |