|
2020-10-16
§
|
| 08:09 |
<elukey> |
reboot stat1005/stat1008 to pick up correct GPU settings |
[production] |
| 08:08 |
<ema> |
upload@ulsfo: upgrade varnish to 6.0.6-1wm2, restart varnishkafka-webrequest T264074 |
[production] |
| 07:59 |
<ema> |
text@ulsfo: upgrade varnish to 6.0.6-1wm2, restart varnishkafka instances T264074 |
[production] |
| 07:19 |
<dcausse@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@27d0b01]: cirrus namespace map: Align output columns with table (duration: 04m 22s) |
[production] |
| 07:15 |
<dcausse@deploy1001> |
Started deploy [wikimedia/discovery/analytics@27d0b01]: cirrus namespace map: Align output columns with table |
[production] |
| 06:57 |
<XioNoX> |
enable cr2-eqdfw:xe-0/1/2 |
[production] |
| 02:14 |
<eileen> |
civicrm revision changed from 585eb835d8 to 3c3dcf80ae, config revision is f76d7849bc |
[production] |
| 01:01 |
<ryankemper> |
Cleaning up a dangling no-longer-puppet-managed udev elasticsearch-readahead rule across all cirrus instances: `sudo cumin -b 36 C:profile::elasticsearch::cirrus 'sudo rm -fv /etc/udev/rules.d/elasticsearch-readahead.rules && sudo /sbin/udevadm control --reload && sudo /sbin/udevadm trigger'` |
[production] |
| 00:56 |
<cdanis@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 00:56 |
<cdanis@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
|
2020-10-15
§
|
| 23:49 |
<ryankemper> |
Began in-place reindex of `eqiad`, `codfw`, and `cloudelastic`. Running on `ryankemper@mwmaint2001` under tmux sessions `inplace_reindex_[eqiad, codfw, cloudelastic]` |
[production] |
| 23:00 |
<krinkle@deploy1001> |
Synchronized wmf-config/env.php: I245e84e0b8c (duration: 01m 10s) |
[production] |
| 22:09 |
<cdanis> |
previous sre.network.cf invocation was a no-op; just checking status |
[production] |
| 22:08 |
<cdanis@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 22:08 |
<cdanis@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 22:06 |
<mutante> |
depooled remaining wtp* servers in codfw. old parsoid servers, new servers are parse2* (T265558) |
[production] |
| 22:05 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,name=wtp2020.codfw.wmnet |
[production] |
| 22:05 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,name=wtp201[6-9].codfw.wmnet |
[production] |
| 21:35 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,name=wtp201[0-5].codfw.wmnet |
[production] |
| 20:27 |
<cdanis@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 20:27 |
<cdanis@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 19:46 |
<ebernhardson@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@88e1283]: spark: fix handling of unpartitioned data sources (duration: 06m 22s) |
[production] |
| 19:43 |
<marxarelli> |
all wikis promoted to 1.36.0-wmf.13 (T263179) |
[production] |
| 19:39 |
<ebernhardson@deploy1001> |
Started deploy [wikimedia/discovery/analytics@88e1283]: spark: fix handling of unpartitioned data sources |
[production] |
| 19:33 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.13 |
[production] |
| 19:30 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:23 |
<robh@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:20 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.11/extensions/DiscussionTools/: Correctly generate timezone abbreviations for parsing (T265500) (duration: 01m 29s) |
[production] |
| 19:16 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.13/extensions/DiscussionTools/: Correctly generate timezone abbreviations for parsing (T265500) (duration: 01m 51s) |
[production] |
| 19:14 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.13/extensions/Echo/: Drop text indent in modern Vector (T264339) (duration: 01m 51s) |
[production] |
| 19:09 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.13/skins/Vector/: Vertically align personal tools (T264339) (duration: 01m 43s) |
[production] |
| 19:07 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.13/extensions/WikimediaEvents/: Revert "clientError: Adds is_logged_in tag to aid filtering" (T256173) (duration: 01m 58s) |
[production] |
| 19:04 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.13/extensions/UploadWizard/: Work around LESS calculating calc() values wrong (T265560) (duration: 02m 07s) |
[production] |
| 18:32 |
<mutante> |
depooling wtp2005 through wtp2009 (parsoid, old server generation) T265558 |
[production] |
| 18:32 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: dc=codfw,name=wtp200[6-9].codfw.wmnet |
[production] |
| 18:07 |
<mutante> |
mx1001/mx2001: made previous live hack official and added benefactors@wikipedia alias, re-enabling puppet |
[production] |
| 17:51 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 17:46 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 17:19 |
<volans@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 17:17 |
<jbond42> |
deleteing old pcc reports in compiler1002 to free disk space |
[production] |
| 17:12 |
<volans@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 17:06 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'zotero' for release 'staging' . |
[production] |
| 17:05 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 17:00 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'staging' . |
[production] |
| 16:58 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'push-notifications' for release 'main' . |
[production] |
| 16:57 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 16:56 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 16:54 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'mathoid' for release 'staging' . |
[production] |
| 16:51 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . |
[production] |
| 16:50 |
<jayme@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'eventgate-analytics' for release 'canary' . |
[production] |