Be smart. Think open source.
$ cf logs $ cf events $ cf restage
$ cf curl /v2/apps $ cf curl -X DELETE /v2/apps/233f....
$ kubectl get pods -n scf $ kubectl get pods -n uaa
If something is acting strange, or health status is not 1/1, go inside the pod and check monit
Try a recreation of the Pod having a problem
$ kubectl delete pod xxxx
api/0:/var/vcap/jobs/cloud_controller_ng# monit status | head The Monit daemon 5.2.5 uptime: 7d 0h 50m Process 'crond' status running monitoring status monitored
api/0:/var/vcap/jobs/cloud_controller_ng# tail /var/vcap/sys/log/nginx_cc/nginx.error.log 2019/02/15 15:45:40 [warn] 3149#0: *325777 send() to syslog failed while logging request, client: 10.244.4.6, server: _, request: "GET /internal/v4/syslog_drain_urls?batch_size=1000&next_id=0 HTTP/1.1", upstream: "http://unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock/internal/v4/syslog_drain_urls?batch_size=1000&next_id=0", host: "api.scf.svc.cluster.local:9023" 2019/02/15 15:45:50 [error] 3149#0: send() failed (111: Connection refused) 2019/02/15 15:45:50 [error] 3149#0: send() failed (111: Connection refused) 2019/02/15 15:45:50 [warn] 3149#0: *325781 send() to syslog failed while logging request, client: 10.244.5.6, server: _, request: "HEAD /v2/info HTTP/1.1", upstream: "http://unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock/v2/info", host: "api-0:9022"
If a replicaset is giving problems, try to set the replicas to #1
$ kubectl scale statefulset/mysql --replicas 1
troubleshoot on one single replicaset rather than on multiple instances