|
2020-07-14
§
|
| 05:44 |
<James_F> |
Created WMF-deployed REL1_35 branches with `./branch.py --branchpoint 7d04152db4f8ea9a459511bed8117101d9bb4602 --bundle base --core-version 1.35.0-beta --task T256376 REL1_35` |
[releng] |
| 05:17 |
<James_F> |
Zuul: parameter_functions: VisualEditor and TemplateData are now tarballed (no-op) |
[releng] |
| 05:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Decrease a bit db1088 load', diff saved to https://phabricator.wikimedia.org/P11891 and previous config saved to /var/cache/conftool/dbconfig/20200714-051551-marostegui.json |
[production] |
| 05:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1131 for HW maintenance', diff saved to https://phabricator.wikimedia.org/P11890 and previous config saved to /var/cache/conftool/dbconfig/20200714-050931-marostegui.json |
[production] |
| 05:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1093 from api', diff saved to https://phabricator.wikimedia.org/P11889 and previous config saved to /var/cache/conftool/dbconfig/20200714-050912-marostegui.json |
[production] |
| 05:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Promote db1093 to s6 master and remove read-only from s6 T257253', diff saved to https://phabricator.wikimedia.org/P11888 and previous config saved to /var/cache/conftool/dbconfig/20200714-050157-marostegui.json |
[production] |
| 05:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Set s6 as read-only for maintenance T257253', diff saved to https://phabricator.wikimedia.org/P11887 and previous config saved to /var/cache/conftool/dbconfig/20200714-050039-marostegui.json |
[production] |
| 05:00 |
<marostegui> |
Starting s6 failover from db1131 to db1093 - T257253 |
[production] |
| 04:59 |
<James_F> |
1.35.0-wmf.41 branched at 7d04152db4f8ea9a459511bed8117101d9bb4602 |
[production] |
| 04:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1078', diff saved to https://phabricator.wikimedia.org/P11886 and previous config saved to /var/cache/conftool/dbconfig/20200714-043907-marostegui.json |
[production] |
| 04:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1093 in preparation for failover', diff saved to https://phabricator.wikimedia.org/P11885 and previous config saved to /var/cache/conftool/dbconfig/20200714-041548-marostegui.json |
[production] |
| 04:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1130', diff saved to https://phabricator.wikimedia.org/P11884 and previous config saved to /var/cache/conftool/dbconfig/20200714-041440-marostegui.json |
[production] |
| 04:01 |
<andrewbogott> |
added a wildcard *.wmflabs.org domain pointing at the domain proxy in project-proxy |
[admin] |
| 04:00 |
<andrewbogott> |
shortened the ttl on .wmflabs.org. to 300 |
[admin] |
| 01:23 |
<ryankemper> |
Started long-running Elasticsearch reindex of `eqiad`, `codfw`, and `cloudelastic`. tmux session `reindex` under `ryankemper` on `mwmaint1002` |
[production] |
| 01:20 |
<cdanis> |
❌cdanis@lvs1015.eqiad.wmnet ~ 🕤🍺 sudo systemctl restart pybal.service |
[production] |
| 01:15 |
<cdanis> |
✔️ cdanis@lvs1016.eqiad.wmnet ~ 🕘🍺 sudo systemctl restart pybal.service |
[production] |
| 01:14 |
<cdanis> |
✔️ cdanis@lvs2009.codfw.wmnet ~ 🕘🍺 sudo systemctl restart pybal.service |
[production] |
| 01:01 |
<cdanis> |
✔️ cdanis@lvs2010.codfw.wmnet ~ 🕘🍺 sudo systemctl restart pybal.service |
[production] |
|
2020-07-13
§
|
| 23:06 |
<mutante> |
releases* delete /usr/local/sbin/sync-* scripts created by rsync::quickdatacopy and let puppet recreate the ones still needed |
[production] |
| 22:27 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: I80ca62643f5c (duration: 00m 58s) |
[production] |
| 22:12 |
<bd808> |
Add codesearch.wmflabs.org (T256206) |
[redirects] |
| 21:04 |
<Amir1> |
create codesearch.wmcloud.org dns proxy (T256206) |
[codesearch] |
| 20:53 |
<Reedy> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/612364 |
[releng] |
| 20:17 |
<milimetric> |
deployed weekly train with two oozie job bugfixes and rename to pageview_actor table |
[analytics] |
| 20:12 |
<ebernhardson@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@1edde21]: airflow: ship_to_es: Implement multi-index understanding (duration: 00m 29s) |
[production] |
| 20:12 |
<ebernhardson@deploy1001> |
Started deploy [wikimedia/discovery/analytics@1edde21]: airflow: ship_to_es: Implement multi-index understanding |
[production] |
| 20:03 |
<mutante> |
rsynced reprepro data from releases1001 to releases1002, releases2002 |
[production] |
| 19:50 |
<eileen> |
disable target smart job process-control config revision is b00e7680ca |
[production] |
| 19:48 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@de0a1f1] (thin): Regular analytics weekly train THIN [analytics/refinery@de0a1f1] (duration: 00m 07s) |
[production] |
| 19:47 |
<milimetric@deploy1001> |
Started deploy [analytics/refinery@de0a1f1] (thin): Regular analytics weekly train THIN [analytics/refinery@de0a1f1] |
[production] |
| 19:47 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@de0a1f1]: Regular analytics weekly train [analytics/refinery@de0a1f1] (duration: 06m 41s) |
[production] |
| 19:42 |
<joal> |
Deploy refinery with scap |
[analytics] |
| 19:41 |
<milimetric@deploy1001> |
Started deploy [analytics/refinery@de0a1f1]: Regular analytics weekly train [analytics/refinery@de0a1f1] |
[production] |
| 19:39 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:33 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: I1a12124f1811e9a (duration: 00m 57s) |
[production] |
| 19:24 |
<joal> |
Drop pageview_actor_hourly and replace it by pageview_actor |
[analytics] |
| 18:53 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T248343 Don't use the 'zeroconf' configuration for VisualEditor (duration: 00m 55s) |
[production] |
| 18:43 |
<dcausse> |
BACON done |
[production] |
| 18:40 |
<dcausse@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T257745: Add rollbacker to elwiki (duration: 00m 56s) |
[production] |
| 18:26 |
<dcausse@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T250810: Set proper language code for some wikis (duration: 00m 56s) |
[production] |
| 18:26 |
<joal> |
Kill pageview_actor_hourly and unique_devices_per_project_family jobs to copy backfilled data |
[analytics] |
| 18:18 |
<dcausse@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T256928: Scale largest shards to be closer to 30GB (duration: 00m 56s) |
[production] |
| 16:17 |
<arturo> |
icinga downtime cloudcontrol[1003-1005].wikimedia.org for 1h for galera database movements |
[admin] |
| 16:17 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:17 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:56 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/Wikibase.php: [[gerrit:610265|Load WikibaseClient using extension registration in beta (T257435)]] (duration: 00m 55s) |
[production] |
| 15:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1130', diff saved to https://phabricator.wikimedia.org/P11882 and previous config saved to /var/cache/conftool/dbconfig/20200713-155240-marostegui.json |
[production] |
| 15:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1144:3315', diff saved to https://phabricator.wikimedia.org/P11881 and previous config saved to /var/cache/conftool/dbconfig/20200713-154847-marostegui.json |
[production] |
| 15:44 |
<Reedy> |
jjb deployed wikimedia-portals-build to bring in https://gerrit.wikimedia.org/r/612371 |
[releng] |