3901-3950 of 10000 results (47ms)
2016-11-22 §
14:32 <hashar@tin> Synchronized wmf-config: Add missing $wgPropertySuggesterClassifyingPropertyIds for beta (duration: 00m 56s) [production]
14:32 <_joe_> upgraded firejail on all scb nodes in codfw [production]
14:31 <hashar@tin> scap aborted: Add missing $wgPropertySuggesterClassifyingPropertyIds for beta (duration: 09m 32s) [production]
14:22 <hashar@tin> Started scap: Add missing $wgPropertySuggesterClassifyingPropertyIds for beta [production]
14:20 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: mw1171.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=appserver', 'service=apache2']) [production]
14:20 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: mw1170.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=appserver', 'service=apache2']) [production]
14:17 <zeljkof> EU SWAT continues [production]
14:14 <marostegui> Deploy ALTER table db1040 (master) commonswiki.revision - https://phabricator.wikimedia.org/T147305 [production]
14:07 <hashar> European SWAT completed [production]
14:04 <hashar@tin> Synchronized wmf-config/InitialiseSettings.php: Enable RevisionSlider (non BetaFeature) on de,ar,hewiki - T149995 T148646 T150573 (duration: 00m 54s) [production]
12:05 <jynus> retrying schema change on db1040 (page) T151029 [production]
11:57 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1091 (duration: 00m 49s) [production]
11:42 <volans> fixed logrotate on cp1008, removed empty created .1.gz files T151314 [production]
11:16 <marostegui> Deploy ALTER table db1091 commonswiki.revision - T147305 [production]
11:15 <jynus> performing blocking schema change on db1091 (depooled) T151029 [production]
11:07 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1091 (duration: 00m 57s) [production]
10:34 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1059 & db1084 T151029 (duration: 00m 51s) [production]
10:01 <gehel> starting elasticsearch codfw cluster restart for JDK and nginx upgrade [production]
10:00 <gehel> starting elasticsearch cluster restart for JDK and nginx upgrade [production]
09:46 <hashar> Replaced slow Jenkins job operations-puppet-puppetlint-strict in favor of using 'rake test' which runs puppet-lint solely against files changed in HEAD https://gerrit.wikimedia.org/r/322839 [production]
09:44 <jynus> performing blocking schema change on db1084 (depooled) T151029 [production]
09:36 <jynus> performing blocking schema change on db1059 (depooled) T151029 [production]
09:19 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1059 T151029 (duration: 00m 49s) [production]
09:00 <marostegui> Deploy ALTER table db1084 commonswiki.revision - T147305 [production]
08:58 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1084 - T147305 (duration: 00m 53s) [production]
08:33 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1081 - T147305 (duration: 00m 54s) [production]
08:14 <marostegui> Deploy ALTER table db1081 commonswiki.revision - T147305 [production]
08:10 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1081 - T147305 (duration: 00m 50s) [production]
07:23 <marostegui> Reboot db1092 for RAID controller upgrade - T151272 [production]
06:57 <marostegui> Stopping Replication on db2057 for maintenance - T150960 [production]
06:54 <marostegui> Stopping Replication on db1095 for maintenance - T150960 [production]
02:32 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Nov 22 02:32:17 UTC 2016 (duration 4m 18s) [production]
02:27 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.3) (duration: 10m 20s) [production]
02:05 <ejegg> updated SmashPig from 3cbb42fb7138e780bd76a2f17c48983f8f882c57 to 36be698face889cef0be54d9388259e5617e0987 [production]
01:40 <urandom> T151086: RESTBase: Starting 'a' instance Cassandra cleanups, rack 'b', codfw [production]
01:26 <mutante> restbase2011 enabling puppet, initial run after activation with gerrit 322807 [production]
01:14 <mutante> restbast2011 - enabling puppet, running puppet, seeing error about missing secret, disabling puppet again [production]
00:54 <ejegg> updated fundraising tools from 4f54cd815cf9627265c4d004f51c8caffa97be49 to da8092927b1eb3af0cc98c797411bdd857ece3f1 [production]
00:52 <godog> reboot ms-be2025 T151201 [production]
00:47 <bblack> cr[12]-ulsfo - added metric 15 to lvs4002 in policy LVS_import (for real this time) - T151273 [production]
00:45 <bblack> cr[12]-ulsfo - added metric 15 to lvs4002 in policy LVS_import - T151273 [production]
00:35 <bblack> depooled cp4008 (cache_text ulsfo) - T151275 [production]
00:35 <bblack@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp4008.ulsfo.wmnet [production]
00:03 <reedy@tin> Synchronized wmf-config/db-eqiad.php: Depool db1092 after crash T151272 (duration: 00m 59s) [production]
2016-11-21 §
23:53 <robh> db1092, typo in my log! [production]
23:51 <robh> db1095 alerted icinga, non-responsive to serial console (hard crash), rebooting [production]
23:38 <twentyafterfour> disabling puppet on iridium until https://gerrit.wikimedia.org/r/#/c/322791/ lands [production]
23:37 <awight> update civicrm from 3a772366b1fdfd9d5ebd4da1efe7db818401c4e5 to 8c76f437c9a9739ffae2892339a46d62d13fdf61 [production]
23:16 <ejegg> enabled donations queue consumer [production]
23:15 <ejegg> updated fundraising dashboard from af8a493ab9ac5431e0d294e5019ac4e426ac6e08 to 43039fde037aa983d4490da08c8072bb2788e28a [production]