|
2020-03-17
§
|
| 22:49 |
<Amir1> |
warming up cache for Q80M to Q88M for new term store on db1111, db1126, db1104, db1092 (T219123) |
[production] |
| 22:17 |
<bsitzmann@deploy1001> |
Finished deploy [mobileapps/deploy@0adead4]: Update mobileapps to ec6fd6e (duration: 06m 08s) |
[production] |
| 22:11 |
<bsitzmann@deploy1001> |
Started deploy [mobileapps/deploy@0adead4]: Update mobileapps to ec6fd6e |
[production] |
| 22:11 |
<PartyRhino> |
rebooting to sync database and config |
[tools.zppixbot-test] |
| 22:11 |
<PartyRhino> |
accidentally moved (rather than copied) know users database, recovered db |
[tools.zppixbot] |
| 22:04 |
<wm-bot> |
<rhinosf1> rhinosf1@tools.zppixbot-test syncing files & dbs from productions |
[tools.zppixbot-test] |
| 22:04 |
<wm-bot> |
<rhinosf1> rhinosf1@tools.zppixbot-test syncing files |
[tools.zppixbot-test] |
| 21:54 |
<Krinkle> |
krinkle@mw2170$ disable-puppet (Testing for T99740) |
[production] |
| 21:47 |
<wm-bot> |
<rhinosf1> chmod -R g+w |
[tools.zppixbot-test] |
| 21:15 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: WikimediaEditorTasks: Enable Depicts counting (again) (T247874) (duration: 01m 07s) |
[production] |
| 21:10 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: WikimediaEditorTasks: Enable Depicts counting (T247874) (duration: 01m 07s) |
[production] |
| 20:50 |
<mholloway-shell@deploy1001> |
Synchronized php-1.35.0-wmf.23/extensions/WikimediaEditorTasks: Fix revert counting for non-language-specific counters, take 2 (T244974) (duration: 01m 12s) |
[production] |
| 20:33 |
<mutante> |
boron - systemctl start docker-reporter-k8s-images ; systemctl start docker-reporter-releng-images |
[production] |
| 20:31 |
<mutante> |
boron - had degraded systemd state in Icinga - systemctl start docker-reporter-base-images |
[production] |
| 19:54 |
<mutante> |
miscweb1001 - restarted ferm, reverted live hack |
[production] |
| 19:53 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@8db09ed]: Various PCS endpoints additions and fixes T247295 T247096 T244175 (duration: 14m 31s) |
[production] |
| 19:51 |
<mutante> |
miscweb1001 - testing if ferm 80 firewall hole is needed for envoy, temp. disabled puppet, restarted ferm |
[production] |
| 19:49 |
<James_F> |
Deleted deployment-dumps-puppetmaster02 for T241719 |
[releng] |
| 19:48 |
<James_F> |
Deleted deployment-mediawiki-parsoid10, no longer used. |
[releng] |
| 19:45 |
<James_F> |
Restarting deployment-prep's deployment-mediawiki-07 and deployment-mediawiki-09 |
[releng] |
| 19:38 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@8db09ed]: Various PCS endpoints additions and fixes T247295 T247096 T244175 |
[production] |
| 19:01 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q80M (T219123)]], take II (duration: 01m 06s) |
[production] |
| 19:00 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q80M (T219123)]] (duration: 01m 07s) |
[production] |
| 18:59 |
<thcipriani> |
re-enabling puppet agent on deployment-mediawiki-07 The last Puppet run was at Mon Feb 24 18:26:55 UTC 2020 (31711 minutes ago). Puppet is disabled. reason not specified -- nobody on the machine |
[releng] |
| 18:53 |
<ladsgroup@deploy1001> |
Synchronized php-1.35.0-wmf.24/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseItemTermStoreWriter.php: [[gerrit:580390|Do not lock rows when there's no term returned (T247553 T246898)]], To catch the train (duration: 01m 08s) |
[production] |
| 18:50 |
<otto@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 18:45 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 18:45 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 18:41 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) |
[production] |
| 18:39 |
<mutante> |
removing mw1238 through mw1243 - decom with cookbook (T247780 T245099) |
[production] |
| 18:38 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 18:38 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 18:37 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 18:35 |
<dzahn@cumin1001> |
conftool action : set/pooled=inactive; selector: name=mw123[8-9].eqiad.wmnet |
[production] |
| 18:35 |
<dzahn@cumin1001> |
conftool action : set/pooled=inactive; selector: name=mw124[0-3].eqiad.wmnet |
[production] |
| 18:29 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 18:01 |
<bsitzmann@deploy1001> |
Finished deploy [mobileapps/deploy@b6bff94]: Update mobileapps to 3c73ca3 (duration: 06m 06s) |
[production] |
| 18:00 |
<otto@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 17:58 |
<otto@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 17:56 |
<ladsgroup@deploy1001> |
Synchronized php-1.35.0-wmf.23/languages/LanguageConverter.php: [[gerrit:580361|languages: Don't assume in LanguageConverter (T235360)]] (duration: 01m 07s) |
[production] |
| 17:55 |
<bsitzmann@deploy1001> |
Started deploy [mobileapps/deploy@b6bff94]: Update mobileapps to 3c73ca3 |
[production] |
| 17:55 |
<otto@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 17:53 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw124[0-3].eqiad.wmnet |
[production] |
| 17:53 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw123[89].eqiad.wmnet |
[production] |