|
2020-07-06
§
|
| 23:32 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable sidebar instrumentation on test wikipedia (duration: 00m 56s) |
[production] |
| 23:32 |
<eileen> |
process-control config revision is 3fe6753e56 |
[production] |
| 23:22 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Change some zh canonical namespaces. Don't index NS_USER on hywiki (duration: 00m 58s) |
[production] |
| 22:58 |
<eileen> |
tools revision changed from e974147f27 to 73557b8038 |
[production] |
| 22:14 |
<ryankemper@deploy1001> |
Finished deploy [wdqs/wdqs@65502b2]: 0.3.40 (duration: 18m 58s) |
[production] |
| 21:55 |
<ryankemper@deploy1001> |
Started deploy [wdqs/wdqs@65502b2]: 0.3.40 |
[production] |
| 21:52 |
<hashar> |
Upgraded Jenkins on releases1002 and releases2002 # T256978 |
[production] |
| 21:41 |
<mutante> |
upgrading jenkins on releases1001 and releases2001 (T256980) |
[production] |
| 21:37 |
<mutante> |
importing jenkins 2.235.1 into APT repo for both stretch and buster T256980 |
[production] |
| 20:08 |
<eileen> |
tools revision is e974147f27 |
[production] |
| 19:41 |
<qchris> |
Enabling puppet on gerrit1002 again to catch up with puppetmaster. |
[production] |
| 18:56 |
<addshore> |
backport / deploy window done |
[production] |
| 18:55 |
<addshore@deploy1001> |
Synchronized wmf-config: [[gerrit:569263]] T241975 Wikibase: Remove config option wmgUseEntitySourceBasedFederation (duration: 00m 58s) |
[production] |
| 18:54 |
<addshore@deploy1001> |
sync-file aborted: [[gerrit:569263]] (duration: 00m 00s) |
[production] |
| 18:51 |
<addshore@deploy1001> |
Synchronized wmf-config/Wikibase.php: [[gerrit:608944]] T241975 Wikibase: stop using wmgUseEntitySourceBasedFederation (duration: 00m 56s) |
[production] |
| 18:47 |
<addshore@deploy1001> |
Synchronized dblists/wikidataclient.dblist: T254315 Wikidata client wikis: Define entity sources configuration (take 2) [[gerrit:608839]] (duration: 00m 56s) |
[production] |
| 18:45 |
<addshore@deploy1001> |
Synchronized wmf-config: T254315 Wikidata client wikis: Define entity sources configuration (take 2) [[gerrit:608839]] (duration: 00m 58s) |
[production] |
| 18:38 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T256906 T256907 T256909 T254315 [[gerrit:569260]] Commons: Define entity sources configuration (duration: 00m 56s) |
[production] |
| 18:30 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: adffbe6: Enable validation of new signatures (T248632) (duration: 00m 57s) |
[production] |
| 18:24 |
<urbanecm@deploy1001> |
Synchronized wmf-config/abusefilter.php: 8878c60: Add `abusefilter-view` as a default right for the CU log user (T255506) (duration: 00m 55s) |
[production] |
| 18:22 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 1398171: Add arbcom group to plwiki (T256572) (duration: 00m 56s) |
[production] |
| 18:08 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@bb176c2]: update proxy UI to support multiple pre-set domains (duration: 03m 39s) |
[production] |
| 18:04 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@bb176c2]: update proxy UI to support multiple pre-set domains |
[production] |
| 17:54 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Migrate SearchSatisfaction from EventLogging to EventGate on all wikis - T249261 - take 2 (duration: 00m 56s) |
[production] |
| 17:50 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Migrate SearchSatisfaction from EventLogging to EventGate on all wikis - T249261 (duration: 00m 56s) |
[production] |
| 16:09 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Migrate SearchSatisfaction from EventLogging to EventGate on group1 - T249261 (duration: 00m 58s) |
[production] |
| 15:02 |
<jynus> |
removing old snapshots for x1 on dbprov[12]002 |
[production] |
| 14:50 |
<mholloway-shell@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 14:46 |
<mholloway-shell@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 14:44 |
<mholloway-shell@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 14:42 |
<moritzm> |
installing PHP 7.0 security updates |
[production] |
| 14:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1074', diff saved to https://phabricator.wikimedia.org/P11753 and previous config saved to /var/cache/conftool/dbconfig/20200706-143754-marostegui.json |
[production] |
| 14:36 |
<godog> |
reboot ms-be2025 for hw raid software upgrade - T257214 |
[production] |
| 14:28 |
<godog> |
powercycle ms-be2025, no ssh available - T257214 |
[production] |
| 14:14 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 14:09 |
<marostegui> |
Stop MySQL and poweroff db1079 T257216 |
[production] |
| 14:08 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 14:02 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'depool db1136 from main traffic as it is the only s7 api host right now', diff saved to https://phabricator.wikimedia.org/P11752 and previous config saved to /var/cache/conftool/dbconfig/20200706-140217-jynus.json |
[production] |
| 13:56 |
<marostegui> |
Downtime and reboot db1079 after BBU crash |
[production] |
| 13:54 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'depool db1079', diff saved to https://phabricator.wikimedia.org/P11751 and previous config saved to /var/cache/conftool/dbconfig/20200706-135430-jynus.json |
[production] |
| 13:30 |
<marostegui> |
Deploy schema change on s5 codfw master T253276 |
[production] |
| 13:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Reduce es1024 weight in preparation for tomorrow's switchover T255755', diff saved to https://phabricator.wikimedia.org/P11750 and previous config saved to /var/cache/conftool/dbconfig/20200706-132634-marostegui.json |
[production] |
| 13:03 |
<elukey> |
force umount/mount of /mnt/hdfs on an-airflow1001 to unblock dpkg checks (fuse misbehaving, all checks hanging) |
[production] |
| 12:53 |
<elukey> |
kill hanging lsof processes on an-airflow to reduce cpu load |
[production] |
| 12:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1074', diff saved to https://phabricator.wikimedia.org/P11748 and previous config saved to /var/cache/conftool/dbconfig/20200706-124237-marostegui.json |
[production] |
| 12:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1129', diff saved to https://phabricator.wikimedia.org/P11747 and previous config saved to /var/cache/conftool/dbconfig/20200706-124105-marostegui.json |
[production] |