1351-1400 of 1724 results (12ms)
2017-05-11 §
09:46 <ema> cp4010: downgrade varnish to 4.1.5-1wm4 and check frontend transient memory usage [production]
2017-05-09 §
23:13 <mutante> analytics1027 - decom: revoke puppet cert, delete salt key, puppet node clean/deactivate, check icinga removal (T161597) [production]
2017-05-08 §
20:37 <gehel> silencing elasticsearch shard incinga check, recovery after upgrade is going to take a long time - T161908 [production]
2017-05-02 §
16:56 <ppchelko@naos> Finished deploy [restbase/deploy@6adb0f2]: Summary endpoint enhancements. Restart after a check timeout (duration: 07m 56s) [production]
16:48 <ppchelko@naos> Started deploy [restbase/deploy@6adb0f2]: Summary endpoint enhancements. Restart after a check timeout [production]
16:43 <ppchelko@naos> Started deploy [restbase/deploy@6adb0f2]: Summary endpoint enhancements. Restart after a check fail [production]
2017-05-01 §
17:03 <mutante> phab2001 - start/stop phd service - that fixed "systemd state" icinga check, even though phd does not run just like before [production]
2017-04-26 §
13:56 <andrewbogott> disabled instance creation on Horizon via https://gerrit.wikimedia.org/r/#/c/350414/ and on wikitech via a strategic edit in extensions/OpenStackManager/special/SpecialNovaInstance.php [production]
13:54 <gehel> downtime "ElasticSearch health check for shards" checks for logstash and elasticsearch eqiad - T148506 [production]
2017-04-20 §
01:28 <mutante> ran puppet on all (16) Dell R320 via cumin to add CPU frequency check [production]
2017-04-18 §
16:12 <godog> reboot tin to fix cpu mhz issue and check bios settings - T163158 [production]
2017-04-10 §
14:31 <gehel> deploying new psotgresql replication check, might generate a few icinga alerts -T162345 [production]
2017-04-06 §
12:39 <ema> rebooting cp2006 again to check for potential issues bringing up network ifaces / loading intel_uncore T162029 [production]
2017-04-05 §
20:53 <ppchelko@tin> Finished deploy [trending-edits/deploy@475a5c0]: Fix edit scorer (duration: 05m 34s) [production]
20:47 <ppchelko@tin> Started deploy [trending-edits/deploy@475a5c0]: Fix edit scorer [production]
20:44 <ppchelko@tin> Finished deploy [trending-edits/deploy@475a5c0]: Fix edit scorer (duration: 02m 51s) [production]
20:41 <ppchelko@tin> Started deploy [trending-edits/deploy@475a5c0]: Fix edit scorer [production]
2017-03-27 §
08:38 <hashar@tin> Synchronized php-1.29.0-wmf.17/languages/classes/LanguageKk.php: Check for string initialization in lcfirst() for HHVM 3.18 - T161095 (duration: 00m 52s) [production]
2017-03-22 §
15:41 <hashar@tin> Synchronized php-1.29.0-wmf.16/languages/classes/LanguageKk.php: Check for string initialization in ucfirst() to make HHVM 3.18 happy - T161095 (duration: 00m 44s) [production]
15:40 <hashar@tin> Synchronized php-1.29.0-wmf.16/languages/classes/LanguageAz.php: Check for string initialization in ucfirst() to make HHVM 3.18 happy - T161095 (duration: 00m 48s) [production]
15:34 <hashar@tin> Synchronized php-1.29.0-wmf.17/languages/classes/LanguageKk.php: Check for string initialization in ucfirst() to make HHVM 3.18 happy - T161095 (duration: 00m 54s) [production]
15:33 <hashar@tin> Synchronized php-1.29.0-wmf.17/languages/classes/LanguageAz.php: Check for string initialization in ucfirst() to make HHVM 3.18 happy - T161095 (duration: 00m 59s) [production]
2017-03-20 §
20:52 <mutante> DNS - new Wikipedias "khw" (Khowar) and "kbp" (Kabiye) created (T160868) (T160865) ( on ns0/ns1: authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones to trigger template recreation after edit to langs.tmpl) [production]
2017-03-14 §
15:24 <chasemp> silence toolschecker precise job start check in anticipation of removal [production]
2017-03-09 §
16:10 <elukey> remove Piwik/bohrium health check from Varnish cache misc (https://gerrit.wikimedia.org/r/#/c/342007/) [production]
2017-03-08 §
15:19 <elukey> rebooting mw22(5[4-9]|60) as part of sanity check for T155180 [production]
15:08 <elukey> rebooting mw225[123] as part of sanity check for T155180 [production]
2017-03-07 §
14:46 <jynus> restart labsdb1004 for config and data check [production]
2017-03-03 §
23:19 <mutante> icinga: for special external hosts benefactorevents and eventdonations, "submit passive check result for this host" -> "check_tcp -p 80" to avoid "crit hosts" that just don't respond to ICMP (http://www.htmlgraphic.com/nagios-check-host-without-ping/) [production]
17:34 <hashar> CI is mostly recovered. It could not spawn instance anymore. The queue is being processed and will take a while to be completed. Check status on https://integration.wikimedia.org/zuul/ | T159543 [production]
2017-02-21 §
19:32 <demon@tin> scap failed: RuntimeError 2 test canaries had check failures (rerun with --force to override this check) (duration: 15m 00s) [production]
15:40 <godog> restart navtiming ve asset-check statsd-mw-js-deprecate on hafnium to pick up statsd.eqiad.wmnet change - T157022 [production]
2017-02-14 §
21:47 <thcipriani@tin> Synchronized php-1.29.0-wmf.11/includes/libs/rdbms/loadbalancer/LoadBalancer.php: [[gerrit:337669|Type check the APC value in LoadBalancer::doWait()]] (duration: 00m 50s) [production]
2017-02-10 §
15:15 <jynus> temporarily disabling mariadb replication lag checks to deploy new version of the icinga check script [production]
11:10 <godog> restart navtiming ve asset-check statsd-mw-js-deprecate on hafnium to pick up statsd.eqiad.wmnet change - T157022 [production]
2017-01-26 §
14:28 <zfilipin@tin> Synchronized wmf-config/throttle.php: SWAT: [[gerrit:334134|IP Cap Lift for Edit-a-Thon (T156258)]] [[gerrit:334156|[throttle] Her Girl Friday + Lenny Unconference / Editathon in NYC, 2017-01-28 (T156278)]] (duration: 00m 41s) [production]
2017-01-19 §
23:31 <mutante> icinga - replace check command names in puppet_services.cfg for change 333010 [production]
07:28 <dereckson@tin> Synchronized wmf-config/throttle.php: Fix throttle rule for KCES IMR edit-a-thon (duration: 02m 42s) [production]
2017-01-18 §
19:06 <dereckson@tin> Synchronized wmf-config/throttle.php: Add throttle rule for KCES IMR edit-a-thon (T154312) (duration: 00m 39s) [production]
2017-01-06 §
09:12 <ariel@tin> Synchronized wmf-config/throttle.php: Adjust throttle rule for Maharashtra 'Edit Wikipedia' workshop (VNGIASS) (duration: 02m 46s) [production]
2016-12-14 §
19:22 <thcipriani@tin> Synchronized php-1.29.0-wmf.6/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: SWAT: [[gerrit:327258|Follow-up Ic1f1de26: Fix typo in edit tab selector]] (duration: 00m 49s) [production]
2016-12-07 §
16:47 <elukey> running puppet on some mw codfw appservers to check the new config [production]
2016-12-05 §
16:50 <elukey> added nagios process check alarms for varnishakfka-statsv and varnishkafka-eventlogging on cache::text hosts [production]
2016-11-28 §
14:33 <zfilipin@tin> Synchronized wmf-config/throttle.php: SWAT: [[gerrit:323555|[throttle] Exception for #MOWomenOnWikipedia Edit-A-Thon (T151650)]] (duration: 00m 45s) [production]
2016-11-08 §
00:34 <dereckson@tin> Synchronized wmf-config/throttle.php: Nashville Science edit-a-thon (Vanderbilt library) (T150207) (duration: 00m 47s) [production]
2016-11-03 §
23:23 <thcipriani@tin> Synchronized php-1.29.0-wmf.1/extensions/EventBus/EventBus.php: SWAT: [[gerrit:319661|Add logging and check for empty JSON encoded body (T148251)]] (duration: 00m 47s) [production]
2016-10-27 §
13:34 <gehel> maps / postgres replication checks in error after deployment of https://gerrit.wikimedia.org/r/#/c/315271/ (T147194) - replication is working, only check is failing - icinga is silenced [production]
13:33 <gehel> postgres replication checks in error after deployment of https://gerrit.wikimedia.org/r/#/c/315271/ (T147194) - replication is working, only check is failing - icinga is silenced [production]
2016-10-26 §
06:34 <moritzm> repooled mw2098 (was previously down for hardware check) [production]
2016-10-25 §
15:21 <ori> Synchronized wmf-config/throttle.php: I049bd463: Use correct IP for Vanderbilt 2016-10-25 edit-a-thon throttle exception (T149063) (duration: 01m 20s) [production]