Reading in a point cloud
-
Hi, I'm struggling with getting some data in. It is an x:y:z:r:g:b:time style series of points that define a point cloud. This is lifted from a Raspberry Pi 4 with a Realsense scanner that goes via an AWS server, so it will be a live connection. However all attempts to date to input parse or json parse it fall over at the first colon, i just get a 2:! I've tried local text file and uploading it to the web and using Get URL Text to no avail. The file format is like this.
All I need to unblock is how to wrangle the colon! Dumb question sorry guys.
Cheers,
Rob
Data file:
ÿþSending to 217.125.90.25:13000
Message:
2:424:240:21-185:172:451:69:13:01-168:175:460:53:7:01-156:186:489:56:10:0]-145:201,20:42:7:01-128:214:
561:29:5:01-58:163:427:43:49:241-38:160:421:12:19:51-18:154:405:25:30:15117:145:381:16:21:5134:143:375:
46:60:32150:140:369:47:60:35167:139:366:54:66:40183:139:366:65:79:471100:140:367:63:74:451119:142:373:7
0:80:491138:144:379:50:70:411159:147:386:58:70:41[179:149:391:40:53:28I199:151,396:5:11:01116:105:215:2
9:40:141182:101:266:66:80:481-191:179:451:77:16:01-173:176:457:57:9:01-159:184:479:5.12:01-149:199:518
:47:8:01-133:212:552:32:6:01-112:221:574:24:4:0j-64:165:429:69:65:441-44:162:422:22:30:141-24:159:413:5
:9:01-5:152:396:34:40:20112:148:385:20:28:9129:145:377:46:60:32146:142:371:48:59:30162:140:366:53:67:39
179:140:366:55:69:40195:140:366:59:73:431113:142:370,71:83:491132:144:376:56:70:411152:147:383:57:73:47
1179:150:390:52:60:351193:152:395:21:28:121214:159:399:5:9:11175:107:279:25,37:101179:102:267:46:61,261
-198:175:453:75:13:01-178:176:454:62:12:01-162:182:469:54:10:01-152:196:507:46:9:01-138:210:542:34:5:01
-119:221:571:26:5:01-49:164:424:65:76:521-29:162:417:7:14:21-10:155:401:22:26:1117:151:391:22:30:9124:1
47:380:39:47:24[41:144:372:37:46:22157:142:367:41:54:30/74:142:366:56:70:43190:141:365:74:88:581100:142
:369:63:75:431127:145,374:62:761471147:148,381:57:68,401167:150:397:65:75:441197:152:393:39,47:241207:1
54:397:5,11:21172:108:280:23:34:101178:104:270:49:63,341-207:179:459:67:23:1110
Sending to 217.125.90.25:13000
Message:
2:424:240:21-182:176:451,68:11:01-166:1811:463:56:9:01-155:193:494,51:9:1I-142:206:532:39:6:01-126:222:56.26:5:01-55:167:428:55:62:351-35:164:419:18:25:91-15:159:406:7:10:212:154:394:24:29:12120:150:383:32:
6.26:5:01-55:167:428:55:62:351-35:164:419:18:25:91-15:159:406:7:10:212:154:394:24:29:12120:150:383:32:
40:22136:147:376:34:44:20152:144:368:41:54:30169:143:366:44:55:31185:142:365:61:75:451103:143:367:37:49
:201121:145:371:58:70:411140:147:377:64:73:451160:150:384:62:73:461180:152:389:49:58:321202:155:396:15:
24:121177:107,274:30:40:191-215:183:464.100:112:811-186:177:450:76:15:11-171:181:459:55:9:01-157:190:48
2:55:11:91-146:206:522:43:0:01-132,221:561,30:7,01-61:169:430:56:53:291-41:165:420:57:66:421-21:162:411
:8:15:41-2:157:399:18:23:10115:152:387:13:20:6132:151:383:25:33:12148:146:372:35:44:20164:144:367:37:48
:24181:144:365:56:70:43197:144:365:66:80:501115:145:369:60:72:431134:148:375:56:67:381155:151:383:65:77
:481174:152:387:49:60:311195:155:394:29:39:201175:109:276:25:36:141180:105:266:38:57:241-194:179:451:76
:15,0r-175:181:456:56:10:01-116:229:975:22,5:01-46:168:423:62:73:421-27:165:416:13:20:41-7:160:403:7:11
:0144:149:375:25:33:10160:146:367:33:41:18176:145:366:39:51:22193:145:365:63:78:461110:146:367:60:72:41
189:156:392:38:49:25 210:158:397:8:15:4 173:111:279:27:37:13 0:0:0:64:82:44 1
The data is parsed like this:
Id:resx:resy:sequence|x:y:z:r:g:b|x:y:z:r:g:b|....all the points here....|frameCompleteBoolean
Id = sequence in this program
Id identifies which frame the packet belongs to
Resx and resy are the stream resolution in the raspi, multiplying them wont get you the total amount of points as many have been deleted
Sequence indicates the order the frame has in the stream, useful as a few cores are creating frames, we need to reorder them
A single voxel is one of those x:y:z:r:g:b sequences
And at the very end there is a value that can be either 1 or 0, 1 if the packet is terminating the frame or 0 if the packet is just another part of the frame
Basics: a packet != A frame
A frame is comprised of many packets
So the special class in the script assembles the frames back from the packets received -
Ps here is some output from the Unity version
https://www.youtube.com/watch?v=Mq9iD5pauT4&feature=youtu.be
and it looks like this!
-
Hi Rob,
Apologies but I have a raft of questions about this. Can you provide a bit more information about the stream acquisition? how is it coming into Isadora serial, OSC or other? Also what is the frame rate of the data. Are you sure that Isadora can parse the data at the speed it is being streamed? What is the end use of the data - how will it be used/ rendered in Isadora?
My initial thought about this is to look at limiting the data being sent - in terms of number of points and frame rate - perhaps there is a way for you to get the raspi code to format its data output? I would also just concentrate on parsing the x, y, z coordinates of the point cloud and try to strip out the rest. Also looking at your video capture what are the average number of points per frame? I understand that it is variable but it could be that you start with fewer point resolution and build up the point cloud to find a upper limit that will parse.
The device looks wild - so the raspberry pi 3 is in the enclosure along with the realsense camera? someone has spent a lot of time on that!
I am adding here a patch that reads a point cloud data from a text file containing the x,y,z of primitive shapes. (it will display whatever data file has been saved internally to the patch as I have not included multiple data files)
Kind Regards
Russell
-
Hi Rob,
I the file I uploaded to the previous forum entry was not the intended one. So I am reposting a file that may assist you with parsing your data. I did format a javascript actor (from @Juriaan) to parse separate on the ":" of your data set as follows:
function debug(str) {
print(str);
print("\n");
}function main()
{// Get our text argument
var text = arguments[0];
// Split our string with the ":"
var splitted = text.split(":");
// And show it to our debug panel
for (i = 0; i < splitted.length; i++) {
debug(splitted[i]);
}
return splitted;
}It parses the following data set from a text file:
2:424:240:21-185:172:451:69:13:01-168:175:460:53:7:01-156:186:489:56:10:0]-145:201,20:42:7:01-128:214: 561:29:5:01-58:163:427:43:49:241-38:160:421:12:19:51-18:154:405:25:30:15117:145:381:16:21:5134:143:375: 46:60:32150:140:369:47:60:35167:139:366:54:66:40183:139:366:65:79:471100:140:367:63:74:451119:142:373:7 0:80:491138:144:379:50:70:411159:147:386:58:70:41[179:149:391:40:53:28I199:151,396:5:11:01116:105:215:2 9:40:141182:101:266:66:80:481-191:179:451:77:16:01-173:176:457:57:9:01-159:184:479:5.12:01-149:199:518 :47:8:01-133:212:552:32:6:01-112:221:574:24:4:0j-64:165:429:69:65:441-44:162:422:22:30:141-24:159:413:5 :9:01-5:152:396:34:40:20112:148:385:20:28:9129:145:377:46:60:32146:142:371:48:59:30162:140:366:53:67:39 179:140:366:55:69:40195:140:366:59:73:431113:142:370,71:83:491132:144:376:56:70:411152:147:383:57:73:47 1179:150:390:52:60:351193:152:395:21:28:121214:159:399:5:9:11175:107:279:25,37:101179:102:267:46:61,261 -198:175:453:75:13:01-178:176:454:62:12:01-162:182:469:54:10:01-152:196:507:46:9:01-138:210:542:34:5:01 -119:221:571:26:5:01-49:164:424:65:76:521-29:162:417:7:14:21-10:155:401:22:26:1117:151:391:22:30:9124:1 47:380:39:47:24[41:144:372:37:46:22157:142:367:41:54:30/74:142:366:56:70:43190:141:365:74:88:581100:142 :369:63:75:431127:145,374:62:761471147:148,381:57:68,401167:150:397:65:75:441197:152:393:39,47:241207:1 54:397:5,11:21172:108:280:23:34:101178:104:270:49:63,341-207:179:459:67:23:1110
Here is the implementation of that in a patch - it requires additional substantial formatting to extract the x,y,z coordinates and route them meaningfully to a 3D Model Particles actor.
Read XYZ from Text File_demo file 2.izz
Kind Regards
Russell
-
Hi Russell,
Wow, as ever thanks for the reply. Yeah it's a Rasp Pi 4, generating a circa 400 x 400 xy resolution cloud at about 15fps at the moment. So yes there are some concerns about Izzy doing it in realtime. I'm interested in being able to save off paths etc so fidelity is not an absolute requirement. Also, using isadora is an exercise to integrate this with other sources. The cold truth is I'm trying to build up to doing it in Unreal, but having difficulty finding the expertise, so getting some familiarity with the data in the more accessible Isadora helps. Also fun to push the limits.
The Rasp Pi code I'm not able to currently access, however I can speak to the dev, since I agree, if it were more standardised then there are some frameworks which can already read in stuff like this in three.js etc
They've done a really nice design with laser cut aluminium and 3D printed enclosure, it literally switches on with a rocker switch, connects to wifi and starts transmitting.
The attached Unity CSharp script shows what happens, basically it reads in from an intermediary AWS URL, so the data would be coming in as a live stream, not via OSC/serial.
Another project uses the Pi4 as a remote scanner here: https://hackaday.com/2020/03/3...
Thanks for the patch, it's a little late here so I'll get onto this tomorrow (Covid lung pending!)
Best,
Rob
-
Hi, I'm managing to map values from my data to the co-ordinates, this is a brilliant app in general and the point cloud looks fluid and smooth.
In the meantime I'm testing it using the potted text file.
The data stream I need to access has a hole punch every five seconds that it needs to be sent a password
I am looking at using a 'TCP Send Data' Actor hooked up to a Timed Trigger Actor to access the actual stream.
Cheers,
Rob
byte[] bytes = Encoding.UTF8.GetBytes("CONNECTION_REQUEST|" + serverPassword);
udpClient.Send(bytes, bytes.Length, remoteEP);
string linkServerIP = "15.188.142.121";
int linkServerPort = 12000, localClientPort = 13000;while (communicateWithServer) { // Send a hole punching message every 5 secs with password
udpReassembler.sendTokenToServer(linkServer, serverPassword);
Thread.Sleep(5000);
}
