|
2019-04-03
§
|
| 07:40 |
<marostegui> |
DIsable event scheduler on db1115 before restarting - tendril is stuck |
[production] |
| 07:26 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1120 T219493 (duration: 00m 57s) |
[production] |
| 07:25 |
<marostegui> |
Deploy schema change on db1073, labtestwiki - T219887 |
[production] |
| 07:09 |
<marostegui> |
Stop replication in sync on db1120 and db2034 (x1 codfw master) - T219493 |
[production] |
| 07:07 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1120 T219493 (duration: 01m 13s) |
[production] |
| 06:04 |
<_joe_> |
restart varnish backend on cp1085, causing unavailability |
[production] |
| 05:57 |
<marostegui> |
Fix data drifts on bnwikisource on x1 - T219493 |
[production] |
| 05:31 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool pc1007 (duration: 00m 59s) |
[production] |
| 05:23 |
<marostegui> |
Upgrade pc1007 |
[production] |
| 05:23 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool pc1007 for upgrade (duration: 01m 00s) |
[production] |
|
2019-04-02
§
|
| 23:44 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT enwiki: Restrict move-categorypages to +extendedmover/+sysop/+bot T219261 (duration: 00m 58s) |
[production] |
| 23:30 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT Add new WMCS IP range to wgRateLimitsExcludedIps T167432 (duration: 00m 57s) |
[production] |
| 23:27 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT Enable SandboxLink for rowiki T219855 (duration: 00m 56s) |
[production] |
| 23:22 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SDC: Add 'depicts' statements to search index on testcommons (duration: 00m 59s) |
[production] |
| 21:27 |
<andrewbogott> |
rebooting labservices1001 |
[production] |
| 21:16 |
<andrewbogott> |
rebooting labservices1002 |
[production] |
| 20:54 |
<andrewbogott> |
restarting pdns and pdns-recursor on labservices1001 and 1002 in hopes of getting those machines to act a bit less sluggish |
[production] |
| 20:23 |
<krinkle@deploy1001> |
Synchronized php-1.33.0-wmf.24/skins/Vector/includes/: I6e04b512d / T219864 (duration: 00m 59s) |
[production] |
| 20:20 |
<krinkle@deploy1001> |
Synchronized php-1.33.0-wmf.23/skins/Vector/includes/: I6e04b512d / T219864 (duration: 01m 00s) |
[production] |
| 20:16 |
<marxarelli> |
1.33.0-wmf.24 successfully deployed to group0. errors rates look normal (T206678) |
[production] |
| 20:07 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: Group0 to 1.33.0-wmf.24 |
[production] |
| 19:57 |
<dduvall@deploy1001> |
Finished scap: testwiki to php-1.33.0-wmf.24 and rebuild l10n cache (duration: 44m 20s) |
[production] |
| 19:12 |
<dduvall@deploy1001> |
Started scap: testwiki to php-1.33.0-wmf.24 and rebuild l10n cache |
[production] |
| 18:41 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, full deploy T211125 (duration: 20m 49s) |
[production] |
| 18:22 |
<marxarelli> |
cutting mediawiki branch 1.33.0-wmf.24 (T206678) |
[production] |
| 18:22 |
<marxarelli> |
cutting mediawiki branch 1.33.0-wmf.24 |
[production] |
| 18:20 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, full deploy T211125 |
[production] |
| 18:20 |
<ppchelko@deploy1001> |
deploy aborted: Kafka logging pipeline, full deploy T211125 (duration: 00m 03s) |
[production] |
| 18:20 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, full deploy T211125 |
[production] |
| 18:08 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, canary on restbase2010 T211125 (duration: 02m 33s) |
[production] |
| 18:06 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, canary on restbase2010 T211125 |
[production] |
| 17:59 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@2cb53a7] (dev-cluster): Kafka logging pipeline, dev cluster only T211125 (duration: 03m 25s) |
[production] |
| 17:56 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7] (dev-cluster): Kafka logging pipeline, dev cluster only T211125 |
[production] |
| 17:51 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@3dcf328]: Upgrade swagger to v3, attempt 2, T218218 (duration: 20m 47s) |
[production] |
| 17:37 |
<ejegg> |
updated payments-wiki-staging from 793bce1a5f to 15bcb3d1a6 |
[production] |
| 17:30 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@3dcf328]: Upgrade swagger to v3, attempt 2, T218218 |
[production] |
| 17:30 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@3dcf328] (dev-cluster): Upgrade swagger to v3, attempt 2, T218218 (duration: 03m 02s) |
[production] |
| 17:27 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@3dcf328] (dev-cluster): Upgrade swagger to v3, attempt 2, T218218 |
[production] |
| 16:47 |
<XioNoX> |
- replacing accepted-prefix-limit with prefix-limit in eqsin - T211730 |
[production] |
| 16:44 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@6026ad1]: Switch to swagger 3 T218218 (duration: 04m 52s) |
[production] |
| 16:39 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@6026ad1]: Switch to swagger 3 T218218 |
[production] |
| 16:36 |
<XioNoX> |
- replacing accepted-prefix-limit with prefix-limit on esams - T211730 |
[production] |
| 16:12 |
<XioNoX> |
- replacing accepted-prefix-limit with prefix-limit on cr2-eqiad - T211730 |
[production] |
| 16:02 |
<mutante> |
T194174 - bump. started alerting again 2 days ago |
[production] |
| 16:00 |
<mutante> |
icinga - schedule (30d) downtime for kubernetes operational latencies alerts (T219696) on kubernetes1004 |
[production] |
| 15:57 |
<arturo> |
T219626 reimaging cloudcontrol2001-dev again |
[production] |
| 15:55 |
<mutante> |
scandium - systemctl start parsoid-vd was failed (T201366) |
[production] |
| 15:55 |
<herron> |
beginning rolling upgrade of codfw ELK cluster to 5.6.15 T219571 |
[production] |
| 15:52 |
<mutante> |
icinga - re-enabling notifications for scandium. setup task is resolved yet systemd is alerting, should not have been turned off anymore (T201366) |
[production] |
| 15:39 |
<XioNoX> |
repool eqsin - T219847 |
[production] |