Welcome, Guest
Username: Password: Remember me
This forum is dedicated to discussions around storage and networking in cloud computing.

TOPIC: Failed Migration in Advanced Networking

Failed Migration in Advanced Networking 1 year 6 months ago #10420

  • daquinox
  • daquinox's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 3
  • Karma: 0
We are building out a medium sized CloudStack implementation using the newest 3.0.1 version and have run into a hiccup. When trying to migrate from host to host, CloudStack issues the following error:
Resource [Host:10] is unreachable: Host 10: Unable to prepare for migration due to com.cloud.exception.
InternalErrorException: Failed to create vnet 534: java.lang.NullPointerException 
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:457) 
	at com.cloud.utils.script.Script.execute(Script.java:177) 
	at com.cloud.utils.script.Script.execute(Script.java:155) 
	at com.cloud.agent.resource.computing.LibvirtComputingResource.createVnet(LibvirtComputingResource.java:2108) 
	at com.cloud.agent.resource.computing.LibvirtComputingResource.createVlanBr(LibvirtComputingResource.java:2688) 
	at com.cloud.agent.resource.computing.LibvirtComputingResource.execute(LibvirtComputingResource.java:2066) 
	at com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:929) 
	at com.cloud.agent.Agent.processRequest(Agent.java:517) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827) 
	at com.cloud.utils.nio.Task.run(Task.java:79) 
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
	at java.lang.Thread.run(Thread.java:679

Digging a little deeper, it looks like a script on the destination host is failing.
2012-04-20 10:38:30,948 WARN  [resource.computing.LibvirtComputingResource] (agentRequest-Handler-4:null) Exception: /usr/lib64/cloud/agent/./scripts/vm/network/vnet/modifyvlan.sh -v 534 -p null -o add 
java.lang.NullPointerException
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
	at com.cloud.utils.script.Script.execute(Script.java:177)
	at com.cloud.utils.script.Script.execute(Script.java:155)
	at com.cloud.agent.resource.computing.LibvirtComputingResource.createVnet(LibvirtComputingResource.java:2108)
	at com.cloud.agent.resource.computing.LibvirtComputingResource.createVlanBr(LibvirtComputingResource.java:2688)
	at com.cloud.agent.resource.computing.LibvirtComputingResource.execute(LibvirtComputingResource.java:2066)
	at com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:929)
	at com.cloud.agent.Agent.processRequest(Agent.java:517)
	at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827)
	at com.cloud.utils.nio.Task.run(Task.java:79)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)

From what I can tell, the modifyvlan.sh script is being passed "null" as the interface. In older versions of CloudStack, you could define the different interface names via "kvm.public.device" etc, which seems to be missing in 3.0.1. Is this a value I should be defining elsewhere? Thank you.
The administrator has disabled public write access.

Re: Failed Migration in Advanced Networking 1 year 6 months ago #10469

  • stephan
  • OFFLINE
  • Fresh Boarder
  • Posts: 3
  • Karma: 0
The same happened here, after upgrading a 3.0 to a 3.0.1 on our testing environment running on CentOS 6.

I've added kvm.public.device to agent.properties on every node, but this doesn't work. Even, I didn't expect that to work

The agent.properties currently reads
#Storage
#Mon Apr 23 12:39:07 CEST 2012
guest.network.device=cloudbr0
workers=5
private.network.device=cloudbr0
port=8250
resource=com.cloud.agent.resource.computing.LibvirtComputingResource
pod=1
zone=1
guid=219a2ef7-f7cc-385f-bb2d-c77ff53f3f41
kvm.public.device=publicbridge0
cluster=1
public.network.device=publicbridge0
local.storage.uuid=8773d0cf-1162-4376-b83b-f26a79426e2e
host=192.168.154.3
LibvirtComputingResource.id=1

Fresh startup of VM is working, but after trying to migrate (doesn't matter of migrating a single machine or pushing one node into maintenance), results in the following message in agent.log on the target node. The source node doesn't write anything to the log.
2012-04-23 12:46:59,221{GMT} WARN  [resource.computing.LibvirtComputingResource] (agentRequest-Handler-4:) Exception: /usr/lib64/cloud/agent/./scripts/vm/netwo
rk/vnet/modifyvlan.sh -v 1898 -p null -o add 
java.lang.NullPointerException
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
        at com.cloud.utils.script.Script.execute(Script.java:177)
        at com.cloud.utils.script.Script.execute(Script.java:155)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.createVnet(LibvirtComputingResource.java:2108)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.createVlanBr(LibvirtComputingResource.java:2688)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.execute(LibvirtComputingResource.java:2066)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:929)
        at com.cloud.agent.Agent.processRequest(Agent.java:517)
        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827)
        at com.cloud.utils.nio.Task.run(Task.java:79)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2012-04-23 12:46:59,221 WARN  [resource.computing.LibvirtComputingResource] (agentRequest-Handler-4:null) Exception: /usr/lib64/cloud/agent/./scripts/vm/networ
k/vnet/modifyvlan.sh -v 1898 -p null -o add 
java.lang.NullPointerException
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
        at com.cloud.utils.script.Script.execute(Script.java:177)
        at com.cloud.utils.script.Script.execute(Script.java:155)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.createVnet(LibvirtComputingResource.java:2108)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.createVlanBr(LibvirtComputingResource.java:2688)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.execute(LibvirtComputingResource.java:2066)
        at com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:929)
        at com.cloud.agent.Agent.processRequest(Agent.java:517)
        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827)
        at com.cloud.utils.nio.Task.run(Task.java:79)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
libvir: RPC error : Cannot write data: Broken pipe
libvir: error : invalid argument: virStoragePoolLookupByUUIDString

I was hoping to get this pod productive, but that behaviour is a real show stopper.
Would be nice, if someone could point me to a solution.
The administrator has disabled public write access.

Re: Failed Migration in Advanced Networking 1 year 6 months ago #10508

  • yasker
  • yasker's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 1
  • Karma: 0
This issue should have been fixed now.

See: bugs.cloudstack.org/browse/CS-14618
Last Edit: 1 year 6 months ago by yasker.
The administrator has disabled public write access.

Re: Failed Migration in Advanced Networking 1 year 6 months ago #10512

  • stephan
  • OFFLINE
  • Fresh Boarder
  • Posts: 3
  • Karma: 0
Yes, I also stumbled upon that bugtracker entry. But as there's no way (at least I found no) to get that patched into a running 3.0.1, I need to wait until 3.0.2 is getting packed.
The administrator has disabled public write access.
Time to create page: 0.334 seconds

Open@Citrix

Citrix supports the open source community via developer support and evangeslism. We have a number of developers and evangelists that participate actively in the open source community in Apache Cloudstack, OpenDaylight, Xen Project and XenServer. We also conduct educational activities via the Build A Cloud events held all over the world. 

Connect