Skip to content

Nvmeof service is not coming up when deployed and deleted and deployed again with automate metadata pool #1824

Description

@jprakash-redhat

I have deployed nvmeof service in 2 gws using

ceph orch apply nvmeof --group group1 --placement="ceph-jp-upstream1-lgfntu-node6 ceph-jp-upstream1-lgfntu-node7"

The service is up and running. Created subsystems and namespaces, everything looks fine
Now I have deleted nvmeof service using

ceph orch rm nvmeof.nvmeof.group1 --force

nvmeof service is deleted and no stale entries in ceph orch ls comand output

Now again I tried to deploy nvmeof service in 4 gws using

ceph orch apply nvmeof --group gw_group1 --placement='ceph-jp-upstream1-lgfntu-node6 ceph-jp-upstream1-lgfntu-node7 ceph-jp-upstream1-lgfntu-node8 ceph-jp-upstream1-lgfntu-node9'

Now service is not coming up and status shows as failed

[ceph: root@ceph-jp-upstream1-lgfntu-node1-installer /]# ceph orch ls
NAME                       PORTS                   RUNNING  REFRESHED  AGE  PLACEMENT                                                                                                                    
alertmanager               ?:9093,9094                 1/1  7s ago     3h   count:1                                                                                                                      
ceph-exporter              ?:9926                      9/9  8s ago     3h   *                                                                                                                            
crash                                                  9/9  8s ago     3h   *                                                                                                                            
grafana                    ?:3000                      1/1  7s ago     3h   count:1                                                                                                                      
mgr                                                    2/2  8s ago     3h   label:mgr                                                                                                                    
mon                                                    3/3  8s ago     3h   label:mon                                                                                                                    
node-exporter              ?:9100                      9/9  8s ago     3h   *                                                                                                                            
nvmeof.nvmeof.gw_group1    ?:4420,5500,8009,10008      0/4  8s ago     19m  ceph-jp-upstream1-lgfntu-node6;ceph-jp-upstream1-lgfntu-node7;ceph-jp-upstream1-lgfntu-node8;ceph-jp-upstream1-lgfntu-node9  
osd.all-available-devices                               12  8s ago     3h   *                                                                                                                            
prometheus                 ?:9095                      1/1  7s ago     3h   count:1                                                                                                                      
[ceph: root@ceph-jp-upstream1-lgfntu-node1-installer /]# 
[ceph: root@ceph-jp-upstream1-lgfntu-node1-installer /]# ceph orch ps | grep "nvme" 
nvmeof.nvmeof.gw_group1.ceph-jp-upstream1-lgfntu-node6.wzolvy  ceph-jp-upstream1-lgfntu-node6            *:5500,4420,8009,10008  error            20s ago  25m        -        -  <unknown>              <unknown>     <unknown>     
nvmeof.nvmeof.gw_group1.ceph-jp-upstream1-lgfntu-node7.lrccbw  ceph-jp-upstream1-lgfntu-node7            *:5500,4420,8009,10008  error            20s ago  25m        -        -  <unknown>              <unknown>     <unknown>     
nvmeof.nvmeof.gw_group1.ceph-jp-upstream1-lgfntu-node8.ndpxpo  ceph-jp-upstream1-lgfntu-node8            *:5500,4420,8009,10008  error            20s ago  25m        -        -  <unknown>              <unknown>     <unknown>     
nvmeof.nvmeof.gw_group1.ceph-jp-upstream1-lgfntu-node9.dhitzl  ceph-jp-upstream1-lgfntu-node9            *:5500,4420,8009,10008  error            20s ago  25m        -        -  <unknown>              <unknown>     <unknown>     
[ceph: root@ceph-jp-upstream1-lgfntu-node1-installer /]#

journalctl_logs_16th_mar_25.txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    🆕 New

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions