801-850 of 1724 results (17ms)
2022-11-01 §
13:13 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:851618|viwiki: Increase autoconfirmed edit count to 10 (T322105)]] (duration: 10m 35s) [production]
13:03 <urbanecm@deploy1002> urbanecm and stang: Backport for [[gerrit:851618|viwiki: Increase autoconfirmed edit count to 10 (T322105)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet [production]
13:02 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:851618|viwiki: Increase autoconfirmed edit count to 10 (T322105)]] [production]
2022-10-31 §
20:17 <cjming@deploy1002> Finished scap: Backport for [[gerrit:851109|Update sample rate for edit attempt stream to 1 for group 0. (T312016)]] (duration: 04m 17s) [production]
20:13 <cjming@deploy1002> cjming and cjming: Backport for [[gerrit:851109|Update sample rate for edit attempt stream to 1 for group 0. (T312016)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet [production]
20:12 <cjming@deploy1002> Started scap: Backport for [[gerrit:851109|Update sample rate for edit attempt stream to 1 for group 0. (T312016)]] [production]
2022-10-11 §
07:17 <ryankemper> [Elastic] Forcing recheck of elastic settings check alerts; expecting a bit of noise as the alerts resolve (hopefully) [production]
2022-10-07 §
17:51 <ryankemper> [Elastic] Updated list of cross-cluster remote seeds for all eqiad/codfw elastic clusters; should resolve `ElasticSearch setting check` alerts [production]
16:42 <brennen@deploy1002> Finished scap: Backport for [[gerrit:840180|Check whether title actually exists (T319798)]] (duration: 05m 47s) [production]
16:36 <brennen@deploy1002> brennen and brennen: Backport for [[gerrit:840180|Check whether title actually exists (T319798)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet [production]
16:36 <brennen@deploy1002> Started scap: Backport for [[gerrit:840180|Check whether title actually exists (T319798)]] [production]
2022-09-29 §
20:58 <ryankemper> T313431 Updated cross-cluster seed conf with new masters; should resolve the settings check alerts [production]
2022-09-07 §
13:38 <samtar@deploy1002> Synchronized php-1.39.0-wmf.27/extensions/GrowthExperiments/modules/ext.growthExperiments.MentorDashboard.Vue/components/MenteeOverview/MenteeFiltersForm.vue: Backport: [[gerrit:830199|Mentee overview(vue): prevent clicks on more recent edit buttons to submit the filters (T316926)]] (duration: 04m 07s) [production]
2022-09-01 §
20:49 <thcipriani@deploy1002> Finished scap: Backport for [[gerrit:828616|Remove Vector grid config (T313559)]], [[gerrit:829043|Disable sticky header edit experiment for idwiki, viwki (T315264)]] (duration: 05m 44s) [production]
20:44 <thcipriani@deploy1002> thcipriani and cjming and bwang: Backport for [[gerrit:828616|Remove Vector grid config (T313559)]], [[gerrit:829043|Disable sticky header edit experiment for idwiki, viwki (T315264)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet [production]
20:44 <thcipriani@deploy1002> Started scap: Backport for [[gerrit:828616|Remove Vector grid config (T313559)]], [[gerrit:829043|Disable sticky header edit experiment for idwiki, viwki (T315264)]] [production]
2022-08-31 §
00:14 <ryankemper> T316719 First elastic host upgraded properly. Cancelling cookbook to kick off a new rolling upgrade that will go 3 nodes at a time (first run was just one host as a sanity check) [production]
2022-08-18 §
13:37 <samtar@deploy1002> scap failed: average error rate on 5/9 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) [production]
07:39 <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.25/extensions/GrowthExperiments/modules/ext.growthExperiments.HelpPanel/SuggestedEditsGuidance.js: 520cd7b78631f993681a77e1baa7a77f9b5d0961: Fix structured task restriction check (T315516) (duration: 03m 17s) [production]
2022-08-17 §
21:16 <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.25/extensions/FlaggedRevs: Backport: [[gerrit:824171|Remove indexExists check for page_name_title index]] (duration: 03m 12s) [production]
2022-08-15 §
21:54 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:823229|Revert "Revert "Enable sticky header edit A/B test for idwiki + viwiki""]] (duration: 03m 37s) [production]
20:55 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:823227|Revert "Enable sticky header edit A/B test for idwiki + viwiki"]] (duration: 03m 15s) [production]
20:12 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:821310|Enable sticky header edit A/B test for idwiki + viwiki (T312295)]] (duration: 03m 30s) [production]
2022-08-09 §
08:24 <jynus> starting data check using es1021 and es2021, expect increased read traffic T314559 [production]
2022-08-08 §
20:11 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:817785|Disable sticky header edit A/B test for pilot wikis (T312296)]] (duration: 03m 35s) [production]
2022-08-05 §
15:04 <jbond> upload test-check-clojure to puppet7 component [production]
2022-08-01 §
17:06 <mutante> alert1001 - systemctl restart nsca - pinged by fundraising tech because fundraising hosts have the "passive check is awol" issue again (T196336) [production]
2022-07-26 §
15:30 <_joe_> restarting pybal on lvs1020 to check php 7.4 too [production]
15:25 <_joe_> restarting pybal on lvs1019 to check php 7.4 too [production]
15:23 <_joe_> restarting pybal on lvs2009 to check php 7.4 too [production]
15:18 <_joe_> restarting pybal on lvs2010 to check php 7.4 too [production]
2022-07-14 §
04:32 <oblivian@puppetmaster1001> conftool action : edit; selector: name=ReadOnly,scope=codfw [production]
04:25 <tstarling@puppetmaster1001> conftool action : edit; selector: name=ReadOnly,scope=codfw [production]
2022-07-06 §
20:11 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:811762|Enable sticky header edit A/B test for pilot wikis excluding idwiki/viwiki (T311144)]] (duration: 03m 25s) [production]
2022-06-28 §
13:45 <urbanecm@deploy1002> scap failed: average error rate on 3/9 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) [production]
2022-06-27 §
20:15 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings-labs.php: Config: [[gerrit:808995|Enable sticky header edit test on beta cluster (T310750)]] (duration: 03m 33s) [production]
2022-06-14 §
22:00 <mutante> wtp1026 - manually running '/usr/bin/sudo -u root -- /usr/local/sbin/check-and-restart-php php7.2-fpm 9223372036854775807' [production]
07:16 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:802685|Update $wgVectorMaxWidthOptions to include action=edit (T307725)]] (duration: 03m 36s) [production]
2022-06-09 §
07:43 <jnuche@deploy1002> Synchronized wmf-config/CommonSettings-labs.php: Config: [[gerrit:804017|[beta cluster] Update $wgVectorMaxWidthOptions to include action=edit (T307725)]] (duration: 03m 41s) [production]
2022-06-07 §
16:21 <dduvall@deploy1002> scap failed: RuntimeError scap failed: average error rate on 8/8 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) (duration: 14m 27s) [production]
16:21 <dduvall@deploy1002> scap failed: average error rate on 8/8 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) [production]
2022-05-26 §
01:27 <mutante> T308089 T274463 - gitlab1001 - systemctl start rsync-config-backup-gitlab1003.wikimedia.org - Suceeded - RECOVERY - Check systemd state on gitlab1001 is OK [production]
2022-05-25 §
09:21 <kartik@deploy1002> Synchronized php-1.39.0-wmf.13/extensions/Translate/src/TranslatorInterface/Aid/EditSummariesAid.php: Backport: [[gerrit:798816|EditSummariesAid: Check if title exists before further processing (T309151)]] (duration: 00m 49s) [production]
2022-05-24 §
21:56 <cjming@deploy1002> Synchronized php-1.39.0-wmf.13/extensions/MobileFrontend: Backport: [[gerrit:798811|Follow-up I97c27fd7: Fix after-edit reload in source editor (T309068)]] (duration: 00m 48s) [production]
2022-05-23 §
23:20 <mutante> cumin1001 - systemtl start httpbb_hourly_appserver after deploying gerrit:797533 leads to '+icinga-wm> RECOVERY - Check systemd state on cumin1001 is OK: OK" T116948 [production]
2022-05-22 §
14:14 <krinkle@deploy1002> scap failed: average error rate on 3/8 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) [production]
13:17 <krinkle@deploy1002> scap failed: average error rate on 7/8 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) [production]
2022-05-19 §
18:45 <ryankemper> [WDQS Deploy] Deployed https://gerrit.wikimedia.org/r/793530; ran puppet agent across wdqs* and just kicked off a re-check of the NRPE alerts. We'll see if that clears the Unknown state up [production]
2022-05-18 §
14:39 <jnuche@deploy1002> scap failed: average error rate on 6/8 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) [production]
2022-05-16 §
14:47 <ladsgroup@deploy1002> scap failed: average error rate on 3/8 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org for details) [production]