6301-6350 of 10000 results (101ms)
2023-02-20 §
09:01 <ladsgroup@deploy1002> Synchronized multiversion/MWConfigCacheGenerator.php: Move all of NS-related config out of IS.php to a dedicated file, part II (T308932) (duration: 06m 47s) [production]
08:58 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:54 <ladsgroup@deploy1002> Synchronized wmf-config/core-Namespaces.php: Move all of NS-related config out of IS.php to a dedicated file, part I (T308932) (duration: 16m 10s) [production]
08:54 <ayounsi@cumin1001> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Update wmf-plugin - ayounsi@cumin1001 [production]
08:52 <ayounsi@cumin1001> START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Update wmf-plugin - ayounsi@cumin1001 [production]
08:52 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:44 <jmm@cumin2002> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Mepps out of all services on: 946 hosts [production]
08:43 <jmm@cumin2002> START - Cookbook sre.idm.logout Logging Mepps out of all services on: 946 hosts [production]
08:42 <jmm@cumin2002> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Mepps out of all services on: 1067 hosts [production]
08:42 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:41 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:41 <jmm@cumin2002> START - Cookbook sre.idm.logout Logging Mepps out of all services on: 1067 hosts [production]
08:41 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:41 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:41 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:41 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:41 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:40 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:40 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:40 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:40 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:40 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:40 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:39 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
08:39 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
08:08 <moritzm> updating openjdk-11 on elastic* servers T329957 [production]
07:44 <moritzm> imported jenkins 2.375.3 to thirdparty/ci T330045 [production]
07:41 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
07:41 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
07:40 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
07:40 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
07:40 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
07:39 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
07:39 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
07:39 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
07:27 <Amir1> running migrateTagTemplate.php on all wikis (T329766) [production]
06:40 <hashar> Restarting Gerrit [production]
2023-02-18 §
08:29 <elukey> kill leftover processes of user `mepps` (offboarded) from stat100[4,5] to unblock puppet [production]
08:24 <elukey> delete /var/log/{syslog,messages,user.log).1 on kubestagetcd1005 to free space [production]
08:22 <elukey> delete /var/log/{messages,user.log).1 on kubestageetcd1006 to free space [production]
08:21 <elukey> delete /var/log/syslog.1 on kubestageetcd1006 to free space [production]
2023-02-17 §
22:45 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin1001 - T329957 [production]
22:09 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin1001 - T329957 [production]
22:06 <bking@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin1001 - T329957 [production]
22:05 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin1001 - T329957 [production]
19:30 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host restbase1016.eqiad.wmnet [production]
19:02 <eevans@cumin1001> START - Cookbook sre.hosts.reboot-single for host restbase1016.eqiad.wmnet [production]
18:46 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase10*.eqiad.wmnet: Restarting Cassandra to apply JVM 1.8.0_362 - eevans@cumin1001 [production]
17:49 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2187.codfw.wmnet with OS bullseye [production]
17:49 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]