351-400 of 10000 results (41ms)
2018-10-05 §
09:37 <banyek> disabling puppet on labsdb1009,labsdb1010,labsdb1011 (T203674) [production]
09:36 <banyek> adding wmf-pt-kill_2.2.20-1+wmf2 package for stretch [production]
09:16 <volans> rebooting tegmen, console stuck, possible re-occurrence of T199413 (to be confirmed) [production]
09:12 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Move some wikis for s3 to s5 (duration: 00m 56s) [production]
09:06 <elukey> stop etcdmirror replication on conf2002 [production]
09:05 <_joe_> restarting confd on all nodes in eqiad and esams [production]
08:58 <_joe_> wiped cached values for the read-only etcd SRV record [production]
08:56 <_joe_> read-write connections to etcd only go to codfw now [production]
08:35 <_joe_> reenabling notifications for etcdmirror on conf1005 [production]
08:02 <jynus> start replication on db1069 (x1) [production]
07:54 <jynus> starting replicatios on db1075; db1070, db1070:s3 with disabled gtid [production]
07:50 <jynus> stopping dbstore1001:x1 [production]
07:33 <jynus> chaning s3 master for db1070 [production]
07:28 <jynus> stopping s3 replication on db1070 [production]
07:20 <jynus> stopping x1 replication on db1069 [production]
07:20 <godog> temporarily stop prometheus on bast4001 to finalize data transfer - T179050 [production]
07:19 <jynus> stopping s3 replication on db1075 [production]
07:18 <jynus> stopping s5 replication on db1070 [production]
07:09 <moritzm> installing python3.4/2.7 security updates [production]
05:55 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T205599 - Ic28e00c30 (duration: 00m 57s) [production]
05:53 <_joe_> upgrading python-etcd on conf1004-6, restarting etcdmirror [production]
05:13 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Clarify db1092 status - T205514 (duration: 00m 57s) [production]
04:18 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/includes/libs/filebackend/FileBackendStore.php: T205567 - I75f1eb6dc2cb (duration: 00m 56s) [production]
04:16 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/extensions/CirrusSearch/includes/DataSender.php: I0769c50c (duration: 01m 01s) [production]
00:31 <mutante> LDAP: added user skvjold to group wmf (T204377) [production]
2018-10-04 §
22:51 <ejegg> updated fundraising CiviCRM from 944b954bac to ebc2e0076c [production]
21:27 <XioNoX> bounce phab1001 switch port - T201039 [production]
20:47 <ejegg> updated fundraising CiviCRM from ddf4865650 to 944b954bac [production]
20:23 <mforns@deploy1001> Finished deploy [analytics/refinery@3eb9bf2]: deploying refinery together with refinery-source v0.0.76 (duration: 00m 17s) [production]
20:22 <mforns@deploy1001> Started deploy [analytics/refinery@3eb9bf2]: deploying refinery together with refinery-source v0.0.76 [production]
20:10 <mforns@deploy1001> Finished deploy [analytics/refinery@3eb9bf2]: deploying refinery together with refinery-source v0.0.76 (duration: 14m 04s) [production]
19:56 <mforns@deploy1001> Started deploy [analytics/refinery@3eb9bf2]: deploying refinery together with refinery-source v0.0.76 [production]
19:30 <marxarelli> rise in fatals "Fatal error: entire web request took longer than 60 seconds and timed out in /srv/mediawiki/php-1.32.0-wmf.24/includes/Title.php" [production]
19:26 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.32.0-wmf.24 [production]
19:15 <ppchelko@deploy1001> Finished deploy [cpjobqueue/deploy@6dc89c0]: Bump cirrusSearchLinksUpdate concurrency to 50 (duration: 00m 53s) [production]
19:14 <ppchelko@deploy1001> Started deploy [cpjobqueue/deploy@6dc89c0]: Bump cirrusSearchLinksUpdate concurrency to 50 [production]
18:49 <sbisson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:460202|]] (duration: 00m 59s) [production]
18:24 <XioNoX> bounce lvs1002:eth1 switch port [production]
18:23 <sbisson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:464510|Enable PageTriage/ORES on enwiki (T206149)]] (duration: 01m 01s) [production]
18:21 <bblack> lvs1002: puppet disabled, stopping pybal (fail to 1005) [production]
18:07 <_joe_> disabled notifications for etcd replication lag on conf1005, not in production [production]
17:47 <banyek> repooling labsb1010 (T195747) [production]
17:41 <_joe_> uploaded new python-etcd packages for jessie, stretch [production]
17:38 <XioNoX> asw2-b-eqiad recabling done - T201039 [production]
17:34 <elukey> pool kafka1002 (eventbus) after maintenance [production]
17:22 <elukey> re-enable ircecho after alarms shower [production]
17:15 <andrewbogott> triggering some alerts on labvirt1018 to figure out about alert thresholds [production]
17:06 <elukey> stop ircecho on einstenium - alarms shower [production]
17:02 <gtirloni> tools - published updated toollabs-* Docker images [production]
16:54 <ejegg> updated standalone SmashPig deploy from 82f9d49c23 to 5f21d3f2db [production]