|
2019-04-02
ยง
|
| 21:27 |
<andrewbogott> |
rebooting labservices1001 |
[production] |
| 21:16 |
<andrewbogott> |
rebooting labservices1002 |
[production] |
| 20:54 |
<andrewbogott> |
restarting pdns and pdns-recursor on labservices1001 and 1002 in hopes of getting those machines to act a bit less sluggish |
[production] |
| 20:23 |
<krinkle@deploy1001> |
Synchronized php-1.33.0-wmf.24/skins/Vector/includes/: I6e04b512d / T219864 (duration: 00m 59s) |
[production] |
| 20:20 |
<krinkle@deploy1001> |
Synchronized php-1.33.0-wmf.23/skins/Vector/includes/: I6e04b512d / T219864 (duration: 01m 00s) |
[production] |
| 20:16 |
<marxarelli> |
1.33.0-wmf.24 successfully deployed to group0. errors rates look normal (T206678) |
[production] |
| 20:07 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: Group0 to 1.33.0-wmf.24 |
[production] |
| 19:57 |
<dduvall@deploy1001> |
Finished scap: testwiki to php-1.33.0-wmf.24 and rebuild l10n cache (duration: 44m 20s) |
[production] |
| 19:12 |
<dduvall@deploy1001> |
Started scap: testwiki to php-1.33.0-wmf.24 and rebuild l10n cache |
[production] |
| 18:41 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, full deploy T211125 (duration: 20m 49s) |
[production] |
| 18:22 |
<marxarelli> |
cutting mediawiki branch 1.33.0-wmf.24 (T206678) |
[production] |
| 18:22 |
<marxarelli> |
cutting mediawiki branch 1.33.0-wmf.24 |
[production] |
| 18:20 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, full deploy T211125 |
[production] |
| 18:20 |
<ppchelko@deploy1001> |
deploy aborted: Kafka logging pipeline, full deploy T211125 (duration: 00m 03s) |
[production] |
| 18:20 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, full deploy T211125 |
[production] |
| 18:08 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, canary on restbase2010 T211125 (duration: 02m 33s) |
[production] |
| 18:06 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7]: Kafka logging pipeline, canary on restbase2010 T211125 |
[production] |
| 17:59 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@2cb53a7] (dev-cluster): Kafka logging pipeline, dev cluster only T211125 (duration: 03m 25s) |
[production] |
| 17:56 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2cb53a7] (dev-cluster): Kafka logging pipeline, dev cluster only T211125 |
[production] |
| 17:51 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@3dcf328]: Upgrade swagger to v3, attempt 2, T218218 (duration: 20m 47s) |
[production] |
| 17:37 |
<ejegg> |
updated payments-wiki-staging from 793bce1a5f to 15bcb3d1a6 |
[production] |
| 17:30 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@3dcf328]: Upgrade swagger to v3, attempt 2, T218218 |
[production] |
| 17:30 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@3dcf328] (dev-cluster): Upgrade swagger to v3, attempt 2, T218218 (duration: 03m 02s) |
[production] |
| 17:27 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@3dcf328] (dev-cluster): Upgrade swagger to v3, attempt 2, T218218 |
[production] |
| 16:47 |
<XioNoX> |
- replacing accepted-prefix-limit with prefix-limit in eqsin - T211730 |
[production] |
| 16:44 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@6026ad1]: Switch to swagger 3 T218218 (duration: 04m 52s) |
[production] |
| 16:39 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@6026ad1]: Switch to swagger 3 T218218 |
[production] |
| 16:36 |
<XioNoX> |
- replacing accepted-prefix-limit with prefix-limit on esams - T211730 |
[production] |
| 16:12 |
<XioNoX> |
- replacing accepted-prefix-limit with prefix-limit on cr2-eqiad - T211730 |
[production] |
| 16:02 |
<mutante> |
T194174 - bump. started alerting again 2 days ago |
[production] |
| 16:00 |
<mutante> |
icinga - schedule (30d) downtime for kubernetes operational latencies alerts (T219696) on kubernetes1004 |
[production] |
| 15:57 |
<arturo> |
T219626 reimaging cloudcontrol2001-dev again |
[production] |
| 15:55 |
<mutante> |
scandium - systemctl start parsoid-vd was failed (T201366) |
[production] |
| 15:55 |
<herron> |
beginning rolling upgrade of codfw ELK cluster to 5.6.15 T219571 |
[production] |
| 15:52 |
<mutante> |
icinga - re-enabling notifications for scandium. setup task is resolved yet systemd is alerting, should not have been turned off anymore (T201366) |
[production] |
| 15:39 |
<XioNoX> |
repool eqsin - T219847 |
[production] |
| 15:32 |
<jbond42> |
add cpp-hocon 0.1.6 to jessie-wikimedia/backports |
[production] |
| 15:21 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: VE: Enable mobile section editing A/B test on all remaining wikis T219564 (duration: 00m 51s) |
[production] |
| 15:07 |
<moritzm> |
stopped/disabled ipmievd on cumin2001 |
[production] |
| 14:54 |
<jbond42> |
add leatherman 1.4 to jessie-wikimedia/backports |
[production] |
| 13:44 |
<anomie@mwmaint1002> |
Fixing empty values for 'target_author_actor' in log_search on test wikis and mediawikiwiki for T215525 |
[production] |
| 13:24 |
<volans> |
reboot ms-be2026 to see if that fixes the controller - T219854 |
[production] |
| 13:23 |
<volans@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:23 |
<volans@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:20 |
<volans@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 13:20 |
<volans@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:20 |
<jynus> |
updating puppet compiler facts |
[production] |
| 12:11 |
<arturo> |
icinga downtime toolschecker for 1 month T219243 |
[production] |
| 12:07 |
<hashar> |
contint1001: compressing some MediaWiki debugging logs under /srv/jenkins/builds # T219850 |
[production] |
| 11:42 |
<moritzm> |
restarting parsoid on wtp1025 to pick up openssl update |
[production] |