1101-1150 of 10000 results (57ms)
2018-11-07 §
11:49 <akosiaris> T206439 upload lttoolbox_3.5.0-1+wmf1 [production]
10:06 <hashar> CI: switched operations/puppet job to be based on Stretch ( T208422 ) and to add python3 ( T208873 ) [production]
09:25 <_joe_> run systemctl reset-failed on ms-be1029, had a failed debmonitor session [production]
08:16 <kartik@deploy1001> Finished deploy [cxserver/deploy@6f97d25]: Update cxserver to f9ffd24 (duration: 04m 59s) [production]
08:11 <kartik@deploy1001> Started deploy [cxserver/deploy@6f97d25]: Update cxserver to f9ffd24 [production]
02:55 <ejegg> disabled recurring charge jobs [production]
00:46 <mutante> tegmen - shutting down for renaming and reinstall (T208824) [production]
00:11 <dereckson@deploy1001> Synchronized wmf-config/interwiki.php: Updating interwiki cache (duration: 02m 44s) [production]
2018-11-06 §
23:55 <mutante> cp1084 - network went down, powercycled, probably T203194 [production]
22:49 <ejegg> updated fundraising CiviCRM from e0742d2210 to 769dcf6456 [production]
21:50 <mutante> icinga1001-"MediaWiki EtcdConfig up-to-date" checks were all UNKNOWN because systemd unit update-etcd-mw-config-lastindex was present but service not running. it was turned off in https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/427328/ on purpose. manually ran "systemctl start update-etcd-mw-config-lastindex" and the checks all work (T202782) [production]
21:49 <mutante> icinga1001 - the "MediaWiki EtcdConfig up-to-date" checks were all unknown on the new icinga server, this was because systemd unit update-etcd-mw-config-lastindex was present but service not running. that was turned off in https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/427328/ on purpose. manually ran "systemctl start update-etcd-mw-config-lastindex" to start it and the checks [production]
21:40 <thcipriani@deploy1001> rebuilt and synchronized wikiversions files: Group0 wikis to 1.33.0-wmf.3 [production]
20:19 <thcipriani@deploy1001> Finished scap: testwiki to 1.33.0-wmf.3 and rebuild l10n cache (duration: 34m 06s) [production]
19:45 <thcipriani@deploy1001> Started scap: testwiki to 1.33.0-wmf.3 and rebuild l10n cache [production]
19:44 <thcipriani@deploy1001> Pruned MediaWiki: 1.32.0-wmf.23 (duration: 07m 19s) [production]
18:36 <thcipriani> cutting branch for MediaWiki and extensions version 1.33.0-wmf.3 [production]
17:37 <XioNoX> add vlan-analytics1-a-eqiad interface-range on asw2-a-eqiad [production]
16:47 <XioNoX> enable cr4-ulsfo zayo transport to cr1-codfw [production]
16:23 <akosiaris@deploy1001> scap-helm zotero install --name alextest --set main_app.version=20181019165254-production --set monitoring.enable=true charts/zotero [namespace: zotero, clusters: staging] [production]
16:23 <akosiaris@deploy1001> scap-helm zotero install --set main_app.version=20181019165254-production --set monitoring.enable=true charts/zotero [namespace: zotero, clusters: staging] [production]
16:12 <mutante> einsteinium - temp disabling icinga notifications and puppet, reloading icinga (for extra caution while deploying global NRPE change) [production]
16:07 <mutante> planet1001 - disabling puppet, editing NRPE config, testing allowed_hosts change [production]
16:07 <akosiaris@deploy1001> scap-helm -h finished [production]
16:07 <akosiaris@deploy1001> scap-helm -h cluster staging completed [production]
16:07 <akosiaris@deploy1001> scap-helm -h [namespace: -h, clusters: staging] [production]
15:59 <banyek> updating facts for the puppet compilers [production]
15:37 <akosiaris> create zotero namespace in eqiad, codfw, staging cluster T201611 [production]
15:20 <godog> switch all graphite read traffic to graphite1004 [production]
15:16 <XioNoX> push `lldp port-id-subtype interface-name` to all compatible switches - T208630 [production]
15:14 <jiji> scb1001/scb1002 switched nutcracker redis from rdb1001:6382 to rdb1009:6379 [production]
15:08 <XioNoX> push `lldp port-id-subtype interface-name` to all routers - T208630 [production]
14:40 <jynus_> reducing consistenct temp. on db2048 to avoid lagging [production]
14:30 <moritzm> installing Ruby 2.1 security updates [production]
14:22 <godog> add graphite1004 to graphite cluster for reads [production]
14:21 <moritzm> installing clamav security updates on mendelevium/ticket.wikimedia.org [production]
13:25 <moritzm> restart HHVM on canaries to pick up new curl [production]
13:10 <XioNoX> zeroize asw-b-eqiad (decom) - T208788 [production]
12:33 <moritzm> installing curl security updates [production]
12:17 <moritzm> installing chromium security updates on proton* (new upstream release tested in deployment-prep) [production]
12:12 <zeljkof> EU SWAT finished [production]
12:10 <zfilipin@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:468693|Update several Wikidata-related configs]] (duration: 00m 55s) [production]
11:37 <moritzm> installing Ruby 2.3 security updates [production]
11:37 <moritzm> installing Ruby 2.3 security updates on trusty [production]
11:30 <moritzm> installing Mono security updates [production]
11:23 <moritzm> installing Ruby 1.9 security updates on trusty [production]
11:07 <banyek> stopping replication on db2077 (T208672) [production]
07:25 <_joe_> also restarting on the other eqiad nodes [production]
07:25 <_joe_> restarting tilerator on maps1002 [production]
04:47 <kartik@deploy1001> Finished deploy [cxserver/deploy@ddb0031]: Update cxserver to 17f9a10 (T144467, T198699, T208386) (duration: 05m 26s) [production]