3251-3300 of 10000 results (47ms)
2017-03-26 §
02:20 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.17) (duration: 07m 20s) [production]
2017-03-25 §
22:28 <legoktm@tin> Synchronized wmf-config/: No-op labs only changes https://gerrit.wikimedia.org/r/#/c/344788/ (duration: 00m 52s) [production]
20:08 <Krinkle> Ran mwscript deleteEqualMessages.php on public wikis (T45917) - deleted 5 pages across 5 wikis [production]
13:43 <dereckson@tin> Synchronized wmf-config/throttle.php: Add throttle rule for BordeauxJS (T161402) (duration: 00m 50s) [production]
03:15 <Krinkle> Re-create optimised indexes for xhgui in mongodb on tungsten per https://github.com/perftools/xhgui/tree/v0.7.0#installation (lost after T161196) [production]
02:36 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Mar 25 02:36:46 UTC 2017 (duration 5m 27s) [production]
02:31 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.17) (duration: 12m 22s) [production]
2017-03-24 §
20:03 <krinkle@tin> Synchronized php-1.29.0-wmf.17/StartProfiler.php: touch - T161286 - (symlink) (duration: 00m 42s) [production]
19:55 <krinkle@tin> Synchronized wmf-config/StartProfiler.php: T161286 - include hostname (duration: 00m 49s) [production]
19:33 <krinkle@tin> Synchronized wmf-config/StartProfiler.php: touch - T161286 - hhvm cache maybe? (duration: 00m 43s) [production]
18:10 <ejegg> updated CiviCRM from d3c439f6391b8c6d351caf61eacbcc196dac5d7a to b6c8f3e28ffbd202b1b4221ac07ebc2b44a39069 [production]
17:50 <ebernhardson> restart elasticsearch on relforge100[12] to test reindex api over https [production]
15:27 <jynus> running unscheduled ALTER TABLE on arbcom_cswiki.archive T104756 [production]
13:47 <moritzm> installing freetype security updates on trusty [production]
12:30 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1056 T160415 - T73563 (duration: 00m 44s) [production]
12:13 <marostegui> Start first run of pt-table-checksum on s5 (dewiki) - T161294 [production]
11:18 <godog> upgrade grafana to 4.2.0 on labmon1001 - T161193 [production]
09:39 <godog> pool prometheus100[34] - T148408 [production]
08:23 <marostegui> Deploy schema change s4 db2019 (codfw master) - T160415 [production]
08:01 <ema> upgrading twisted to 16.2.0 on lvs4001 and lvs4002 (ulsfo primaries) T160433 [production]
07:49 <marostegui> Deploy schema change s4 on db1069 and db1056 - T160415 - T73563 [production]
07:48 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1056, repool db1059 T160415 - T73563 (duration: 00m 43s) [production]
07:42 <moritzm> installing git updates on trusty [production]
07:35 <dcausse> cirrus: refresh comp suggest indices in elastic@codfw [production]
07:26 <ema> upgrading twisted to 16.2.0 on lvs4003 and lvs4004 (ulsfo secondaries) T160433 [production]
07:07 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore original weight for db1070, db1071 and db1082 - T137191 (duration: 00m 43s) [production]
06:10 <Krinkle> Removing xhgui.results entries before 1-Dec-2016 finished. Running xhgui->command(compact=>results) now. T161196 [production]
02:31 <Krinkle> Reverted patch - https://gerrit.wikimedia.org/r/#/c/344569/ [production]
02:30 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.17) (duration: 08m 35s) [production]
02:28 <Krinkle> Reminder to incident doc writer: Was difficult figuring out what the last "real" patch was, scap message for SAL is manually written (not says which commit in which repo), and git log contains noise from security patches. We need simple revert options from the flat git tree at /srv/mediawiki [production]
02:26 <Krinkle> Reminder to incident doc writer: Logstash was (and is) not responsive serving Kibana-rendered errors about logstash Service unavailable [production]
02:25 <Krinkle> All apaches are back up [production]
02:24 <krinkle@tin> Synchronized php-1.29.0-wmf.17/extensions/Wikidata: revert (duration: 02m 34s) [production]
02:24 <MaxSem> Killed l10nupdate on tin, was blocking emergency pushes [production]
02:22 <Krinkle> Hard-killed all l10nupdate processes and rm'ed scap lock [production]
02:11 <Krinkle> Removing xhgui.results entries from before 1 December 2016 in MongoDB on tungsten (T161196) [production]
01:45 <mutante> bacula - on helium, attempt to start bacula-director process, attempt to fix permissions on key files as codified in director.pp [production]
01:40 <catrope@tin> Finished scap: Wikidata cherry-picks (with i18n) (duration: 25m 03s) [production]
01:15 <catrope@tin> Started scap: Wikidata cherry-picks (with i18n) [production]
2017-03-23 §
23:26 <Krinkle> Removing xhgui.results entries from before 1 June 2016 (T161196) [production]
23:12 <thcipriani@tin> Synchronized php-1.29.0-wmf.17/extensions/ORES: SWAT: [[gerrit:344555|Stats: Invert "false" thresholds so they are correct]] T161250 (duration: 00m 52s) [production]
23:05 <Pchelolo> update RESTBase to 2536b25c7 - eqiad [production]
22:56 <Pchelolo> update RESTBase to 2536b25c7 - staging [production]
22:39 <Pchelolo> update RESTBase to 2536b25c7 - codfw [production]
21:36 <krinkle@tin> Synchronized wmf-config/StartProfiler.php: (no justification provided) (duration: 00m 53s) [production]
21:05 <ejegg> rolled back payments-wiki to 9622a4bf331426ce85362e2fdb442f226068e971 [production]
21:00 <ejegg> updated payments from 9622a4bf331426ce85362e2fdb442f226068e971 to bb956bf1ea7f0c7a9b08877cce7f25b201bd2bbf [production]
20:16 <thcipriani@tin> rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.17 [production]
19:34 <thcipriani@tin> Synchronized php: Swap symlink for 1.29.0-wmf.17 (duration: 00m 43s) [production]
19:11 <thcipriani@tin> rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.29.0-wmf.17 [production]