2701-2750 of 10000 results (23ms)
2020-09-16 §
08:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es2027 and es2028 with more weight T261717', diff saved to https://phabricator.wikimedia.org/P12601 and previous config saved to /var/cache/conftool/dbconfig/20200916-084916-marostegui.json [production]
08:42 <awight> finished security backport for https://phabricator.wikimedia.org/T262628 [production]
08:41 <awight@deploy1001> Synchronized php-1.36.0-wmf.8/extensions/FileImporter/src/Services/ImportPlanValidator.php: Security patch for T262628 (duration: 00m 59s) [production]
08:41 <XioNoX> asw-c-codfw> request system snapshot slice alternate all-members - T262290 [production]
08:27 <XioNoX> asw-b-codfw> request system snapshot slice alternate all-members - T262290 [production]
08:26 <awight> beginning security backport for https://phabricator.wikimedia.org/T262628 [production]
08:17 <XioNoX> asw-a-codfw> request system snapshot slice alternate all-members - T262290 [production]
08:04 <akosiaris> T187984 Validated that ticket.wikimedia.org works, proceeding with a wider announcement [production]
08:02 <XioNoX> asw2-d-eqiad> request system snapshot slice alternate all-members - T262290 [production]
07:49 <akosiaris> T187984 Switch over ticket.discovery.wmnet to otrs1001 [production]
07:48 <jayme@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
07:44 <jayme@cumin1001> START - Cookbook sre.dns.netbox [production]
07:40 <XioNoX> asw2-c-eqiad> request system snapshot slice alternate all-members - T262290 [production]
07:37 <akosiaris> T187984 Tested inbound email successfully [production]
07:29 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
07:26 <akosiaris> T187984 Tested outbound email, switching inbound email configuration and performing tests [production]
07:26 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es2027 and es2028 with more weight T261717', diff saved to https://phabricator.wikimedia.org/P12600 and previous config saved to /var/cache/conftool/dbconfig/20200916-072614-marostegui.json [production]
07:22 <jayme@cumin1001> START - Cookbook sre.hosts.decommission [production]
07:22 <jayme@cumin1001> END (ERROR) - Cookbook sre.hosts.decommission (exit_code=97) [production]
07:21 <jayme@cumin1001> START - Cookbook sre.hosts.decommission [production]
07:12 <akosiaris> T187984 Disable gravatar in system configuration to avoid leaking agent PII through a 3rd party service [production]
07:03 <akosiaris> T187984 validated that the OTRS installation is functional over SSH [production]
07:02 <akosiaris> T187984 migration script done. Config updates, rebuilds, package upgrades/reinstall and index rebuilds done [production]
06:28 <godog> codfw-prod: bump weight for ms-be2057 - T261633 [production]
06:20 <kart_> Updated cxserver to 2020-08-30-011854-production (T253439, T260557) [production]
06:20 <XioNoX> asw2-b-eqiad> request system snapshot slice alternate all-members - T262290 [production]
06:15 <kartik@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
06:11 <kartik@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
06:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool es2027 and es2028 for the first time with minimum weight T261717', diff saved to https://phabricator.wikimedia.org/P12599 and previous config saved to /var/cache/conftool/dbconfig/20200916-061013-marostegui.json [production]
06:08 <kartik@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . [production]
06:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool es2011 and es2017 after cloning es2027 and es2028', diff saved to https://phabricator.wikimedia.org/P12598 and previous config saved to /var/cache/conftool/dbconfig/20200916-060717-marostegui.json [production]
05:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es2015 to clone es2031 T261717', diff saved to https://phabricator.wikimedia.org/P12597 and previous config saved to /var/cache/conftool/dbconfig/20200916-055535-marostegui.json [production]
05:53 <XioNoX> asw2-a-eqiad> request system snapshot slice alternate all-members - T262290 [production]
05:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es2011 and es2017 after cloning es2027 and es2028', diff saved to https://phabricator.wikimedia.org/P12596 and previous config saved to /var/cache/conftool/dbconfig/20200916-055108-marostegui.json [production]
05:50 <XioNoX> msw1-codfw> request system snapshot slice alternate - T262290 [production]
05:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Add es2027 and es2028 to dbctl T261717', diff saved to https://phabricator.wikimedia.org/P12595 and previous config saved to /var/cache/conftool/dbconfig/20200916-053918-marostegui.json [production]
05:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es2011 and es2017 after cloning es2027 and es2028', diff saved to https://phabricator.wikimedia.org/P12594 and previous config saved to /var/cache/conftool/dbconfig/20200916-053507-marostegui.json [production]
05:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1087 into vslow', diff saved to https://phabricator.wikimedia.org/P12593 and previous config saved to /var/cache/conftool/dbconfig/20200916-052343-marostegui.json [production]
05:22 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es2011 and es2017 after cloning es2027 and es2028', diff saved to https://phabricator.wikimedia.org/P12592 and previous config saved to /var/cache/conftool/dbconfig/20200916-052241-marostegui.json [production]
05:07 <marostegui> Repool labsdb1010 [production]
02:22 <mutante> deneb - sudo systemctl start package_builder_Clean_up_build_directory to fix icinga alert after failed build attempts [production]
2020-09-15 §
23:20 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.9/extensions/FlaggedRevs/backend/FlaggedRevsHooks.php: 1c0b0d161fe1024d6d08a27bbacf5b62c56c9c01: Fix APCOND_FR_NEVERBLOCKED handling (T262970) (duration: 00m 56s) [production]
23:18 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.8/extensions/FlaggedRevs/backend/FlaggedRevsHooks.php: 5beace32a396adfcce46b04e7f969b2f9f9effda: Fix APCOND_FR_NEVERBLOCKED handling (T262970) (duration: 00m 58s) [production]
23:14 <urbanecm@deploy1001> Synchronized wmf-config/flaggedrevs.php: ac8bd3894f2dc8f2735cc9fa7b860af1d91c6707: flaggedrevs: Remove non-existent config options (duration: 00m 58s) [production]
23:07 <urbanecm@deploy1001> Scap failed!: Call to mwscript eval.php stderr: not empty [production]
23:00 <urbanecm@deploy1001> Synchronized wmf-config/abusefilter.php: 62b21d55a8f0a94b8cd268d5024df0cf64013dd5: Revert "Remove abusefilter-view right grant from wmf-config" (T255506) (duration: 00m 59s) [production]
20:44 <brennen> removing extraneous recursive symlink /srv/mediawiki-staging/php-1.36.0-wmf.9/php-1.36.0-wmf.8 [production]
18:32 <Urbanecm> Morning B&C done [production]
18:28 <urbanecm@deploy1001> Synchronized wmf-config/abusefilter.php: 084729b7fd0716f11265f1b37570afc120b27109: Remove abusefilter-view right grant from wmf-config (T255506) (duration: 00m 56s) [production]
18:12 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 1d3456570b80b1d8af1d2b71975496e54f87b24e: Enable MediaWiki client errors on frwiki (T255585) (duration: 00m 57s) [production]