4901-4950 of 10000 results (37ms)
2016-10-03 §
09:28 <marostegui> dbstore2001 going to be reimaged as jessie [production]
09:27 <gehel> rolling restart of elasticsearch codfw cluster for kernel upgrade - T146123 [production]
09:14 <akosiaris> T147173 clean exim queues on mx1001 from backscatter spam [production]
09:08 <akosiaris> clean exim queues on mx1001 from backscatter spam [production]
08:46 <elukey> rebooted compiler02.puppet3-diffs.eqiad.wmflabs (not reachable by Jenkins, pingable from bastions but no ssh available) [production]
08:04 <_joe_> powercycling mw1207 [production]
07:56 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1081 for maintenance - T147113 (duration: 00m 50s) [production]
07:24 <volans> emptying /var/log/debug on dubnium because of disk full (the same data is on syslog) T147173 [production]
06:30 <marostegui> altering S3,S4,S5,S6,S7 user_groups tables in sanitarium to avoid tokudb bug - T146121 [production]
02:28 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.20) (duration: 13m 16s) [production]
2016-10-02 §
07:29 <gehel> silencing wdqs response time alerts, it is flapping, related to traffic - T147130 [production]
04:58 <cwd|afk> updated smash pig from 4b36376f4b206406b5b88661cfcecf1b588d5bcf to efe8720e454b03740e4fb4e846657cc92d5ecf7c [production]
02:27 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.20) (duration: 13m 07s) [production]
2016-10-01 §
11:03 <Amir1> ladsgroup@terbium:~$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki Gautehuus Neuraxıs [production]
02:27 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.20) (duration: 12m 51s) [production]
2016-09-30 §
22:06 <Krinkle> Re-run mwscript deleteEqualMessages.php on all wikis it was previously run on (T45917) [production]
18:49 <ejegg> updated SmashPig from 8ff1950ccd87c649f1748f25e1a0a708c3337206 to 4b36376f4b206406b5b88661cfcecf1b588d5bcf [production]
17:57 <ejegg> updated civicrm from 18e59abac57ba85ee9d9dbd50f9f25df64522974 to e2b5bbfbdaaad29925fc60586ce7a2da8297cc2d [production]
13:33 <yuvipanda> restart grafana-server on labmon1001 [production]
02:33 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Sep 30 02:33:12 UTC 2016 (duration 4m 49s) [production]
02:28 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.20) (duration: 13m 31s) [production]
00:29 <matt_flaschen> Manually updated the DB to fix already-broken cases caused by since-fixed T138310 [production]
00:25 <ejegg> updated civicrm from 637659ee8257562492405385d3fadaee53db998b to 18e59abac57ba85ee9d9dbd50f9f25df64522974 [production]
00:12 <ejegg> updated SmashPig from 3811f0f1c4bed1bd0b02264b5865ae36021cb275 to 8ff1950ccd87c649f1748f25e1a0a708c3337206 [production]
00:12 <ejegg> updated SmashPig // FIXME: var map can't put one thing in two places [production]
00:07 <ejegg> re-enabled donations queue consumer [production]
2016-09-29 §
23:38 <ejegg> updated CiviCRM from 6768613907598c52b0af1ffae35faa9078f15f63 to 637659ee8257562492405385d3fadaee53db998b [production]
23:25 <ejegg> disabled donations queue consumer [production]
23:06 <ejegg> enabled mirroring completed donations queue from payments-wiki [production]
23:03 <ejegg> updated SmashPig from 2169b71016e2deb1114655d0256d3286ff057943 to 3811f0f1c4bed1bd0b02264b5865ae36021cb275 [production]
21:14 <ejegg> updated SmashPig from 077ffcc3c4485f62a2f9c80eb4843ef8c72d0c4f to 2169b71016e2deb1114655d0256d3286ff057943 [production]
18:25 <ejegg> disabled CiviCRM dedupe jobs [production]
18:24 <bd808> https://tools.wmflabs.org/sal/ missing some entries for 2016-09-29; consider https://wikitech.wikimedia.org/wiki/Server_Admin_Log canonical [production]
18:21 <cwd> rolled forward PaymentListeners again [production]
17:54 <cwd> updated smashpig from 0d88feaf8ecab0286e36a91303bb234c68fd6384 to 077ffcc3c4485f62a2f9c80eb4843ef8c72d0c4f [production]
17:26 <cwd> rolled back PaymentListeners [production]
16:52 <ejegg> enabled adyen job runner [production]
16:49 <cwd> updated smashpig from 3458f93599084815da46a2540e9ed762c8b120ce to 40c4a7c664dc53f16943aa0b83f30ab1ce435c15 [production]
16:34 <elukey> executed 'sudo salt -C 'G@cluster:imagescaler and G@site:eqiad' cmd.run 'find /var/log/hhvm/ -type f -user root -exec chown www-data:www-data {} \;' to reduce cronspam [production]
16:33 <elukey> executed 'sudo salt -C 'G@cluster:imagescaler and G@site:codfw' cmd.run 'find /var/log/hhvm/ -type f -user root -exec chown www-data:www-data {} \;' to reduce cronspam [production]
16:32 <urandom> T133395: restbase staging: starting bootstrap of restbase-test2001-b.codfw.wmnet (test of decomm/bootstrap under time-windowed compaction) [production]
15:18 <urandom> T133395: restbase staging: decommissioning restbase-test2001-b.codfw.wmnet (test of decomm/bootstrap under time-windowed compaction) [production]
13:28 <cwd|afk> disabled Adyen job runner [production]
10:29 <hashar> Upgrading Jenkins plugins with zeljkof :] [production]
08:38 <reedy@tin> Synchronized wmf-config/mobile-labs.php: Remove transfers of non existent $wmg variables (duration: 00m 48s) [production]
02:14 <eileen> civicrm upgraded from 86371233a9218a526f85cd76d7008f966042fde7 to 6768613907598c52b0af1ffae35faa9078f15f63 [production]
00:57 <cwd> updated SmashPig from 3458f93599084815da46a2540e9ed762c8b120ce to 3d0a76b5918f280602e5dabeeb0373c1a118590d [production]
2016-09-28 §
23:51 <ejegg> updated SmashPig from d2589271d43d8895fbb928f54f8feb43fcbc43c9 to 3458f93599084815da46a2540e9ed762c8b120ce [production]
22:06 <cwd> PaymentListeners rolled back [production]
21:54 <cwd> PaymentListeners updated from b4d77a991e100f97d98fcd72eaf03940a4e1845d to 21647c8f4b781b74ae2dc4377334410b4eed7 [production]