Ulysse Posted June 11, 2018 Share Posted June 11, 2018 Hi, I am having troubles connecting my Sawyer to a local NTP server. I have a local NTP server on my network that I know to be working : I can query it using some Linux machine and get offset, jitter, etc. I want to connect Sawyer to it so that it is in sync with my development computer (otherwise I can't use MoveIt! nor tf, pretty embarrassing...) I followed the Time and NTP tutorial, booted Sawyer into the FSM, replaced the NTP pool list with the IP address of my NTP server and rebooted into the SDK. However it didn't work, timestamps published by the robot are still off by about 6 hours. For completeness: The robot Intera version 5.2.2 The development computer Intera version is 5.2 I work in France so I am on GMT+2 timezone The `ntpdate` command does not work, it raises the error: server 192.168.212.60, stratum 0, offset 0.000000, delay 0.00000 11 Jun 10:28:25 ntpdate[4829]: no server suitable for synchronization found The `clockdiff` command works and outputs the above offset: host=192.168.212.60 rtt=562(280)ms/0ms delta=-17953355ms/-17953355ms Mon Jun 11 10:54:02 2018 I spent some time on the FSM trying to get this working and I might have disrupted some parameters. For instance, the `ntpdate` was working at first but it no longer does. How can I investigate on why is the robot not connecting to the server ? Normally it should burst on it at boot time as the offset is way bigger than 17 minutes. Thanks for your help. Link to comment Share on other sites More sharing options...
Ian McMahon Posted June 11, 2018 Share Posted June 11, 2018 Hi Ulysse, I'd recommend using some of example settings in the tutorial for your workstation's computer: # editing your workstation's /etc/ntp.conf # remove or comment out your workstation's references to other NTP servers or NTP pools # add backup fake local server server 127.127.1.0 iburst # use local clock of sdk workstation as the server, burst synchronize fudge 127.127.1.0 stratum 10 # "fake" sdk workstation as a stratum 10 NTP server (rather than the default and invalid stratum 16) # set permissions to use your computer as an NTP server restrict -4 default kod notrap nomodify nopeer limited # remove the default "noquery" options to allow others on the LAN to see the NTP server restrict -6 default kod notrap nomodify nopeer limited # ditto # optionally broadcast your IP address to the network broadcast <your>.<ip>.<address>.<here> Of particular interest, you'll need to "fudge" a realistic stratum like 10 to convince other NTP clients to use your clock, unless you actually have a stratum 0 atomic clock. I hope this help! ~ Ian Link to comment Share on other sites More sharing options...
Ulysse Posted June 12, 2018 Author Share Posted June 12, 2018 Please, I've been pulling my hair out on this for two weeks, obviously I tried everything that's on the tutorial. I'm sorry for being rude, I totally understand that you need to post the obvious solutions for other users that might come here. But in the meantime, can you also provide me with a real support ? My issue is not trivial and has been blocking my work for weeks... As a sign of goodwill I tried again the solution you proposed : it still isn't working, the offset keeps oscillating around (minus) 5 hours. Link to comment Share on other sites More sharing options...
Ian McMahon Posted June 12, 2018 Share Posted June 12, 2018 Hi Ulysse, A large portion of the time, these tough technical problems are solved by a simple change somewhere along the way, which is why I started from the beginning with my recommendations. I'm sorry to hear that this issue has been frustrating you, and we will do our best to help. First, a few questions to help me understand your network topology: Are you on a university network? Are your machines connected to the Internet? Which machine is acting as the NTP server? How are you networking your workstation to the Sawyer? You've mentioned that the robot is off by about 5-6 hours in time. If I'm not mistaken, you're located in France in GMT+2. The robot's default timezone is EST, which is currently GMT-4. Is it possible that the timezone of your robot is still set to EST? That would put your robot exactly 6 hours behind. You can check and modify this in the FSM:http://sdk.rethinkrobotics.com/intera/Field_Service_Menu_(FSM)#Menu_Functions Best, ~ Ian Link to comment Share on other sites More sharing options...
Ulysse Posted June 12, 2018 Author Share Posted June 12, 2018 It is an enterprise network on a VLAN, using a separate IP address range: 192.168.212.0/22. There is no internet access as no gateway has been configured (FSM won't let me leave the field empty so I have to type a dummy one). Lastly there is a NTP server on the network whose IP address is 192.168.212.22. I know it is working as my workstation computer is in sync with it. I double-checked the timezone in the FSM and it IS set to Europe/Paris. But I don't think this really matters as ROS timestamps are in nanoseconds since epoch, which is timezone independent. Furthermore the offset isn't exactly 5 hours (I know I wrote 6 earlier on, my bad), it is 5 hours minus ~40 seconds. Thanks for your help. Link to comment Share on other sites More sharing options...
Ian McMahon Posted June 12, 2018 Share Posted June 12, 2018 You mention in your last comment that the NTP server is at 192.168.212.22. In the initial post your workstation configuration file states the server is at 192.168.212.60. Which IP do you have in Sawyer's FSM under NTP Servers? I have a screencrap of the field in question attached. Just so I have my information correct, could you list out the local IP addresses of the robot, user workstation, and NTP servers? And could you post the contents of your workstation's NTP configuration file? Thank you, ~ Ian Link to comment Share on other sites More sharing options...
Ulysse Posted June 13, 2018 Author Share Posted June 13, 2018 Sorry I was confusing, my examples using `ntpdate` and `clockdiff` were to check clock offsets between my workstation and the Sawyer. For clarity (all IP addresses are configured statically): NTP server: 192.168.212.22 Sawyer: 192.168.212.60 Workstation (Ubuntu 16.04 LTS virtual machine running under Oracle VirtualBox on Windows 7 Enterprise): 192.168.212.160 Content of my workstation's `/etc/ntp.conf` file: # ntp.conf: Managed by puppet. # # Enable next tinker options: # panic - keep ntpd from panicking in the event of a large clock skew # when a VM guest is suspended and resumed; # stepout - allow ntpd change offset faster tinker panic 0 disable monitor # Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. restrict -4 default kod nomodify notrap nopeer noquery restrict -6 default kod nomodify notrap nopeer noquery restrict 127.0.0.1 restrict ::1 # Set up servers for ntpd with next options: # server - IP address or DNS name of upstream NTP server # iburst - allow send sync packages faster if upstream unavailable # prefer - select preferrable server # minpoll - set minimal update frequency # maxpoll - set maximal update frequency server 192.168.212.22 iburst prefer server 10.8.113.254 iburst server time.my.company.fr iburst # Driftfile. driftfile /var/lib/ntp/drift Some outputs from my workstation: user@workstation:~$ ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== 192.168.212.22 132.166.192.1 3 u 32 64 0 0.000 0.000 0.000 +10.8.113.254 192.168.8.2 3 u 4 64 7 1.117 4.700 10.553 *my.company 195.220.94.163 2 u 3 64 7 1.154 -2.642 11.157 user@workstation:~$ ntpdate -q 192.168.212.22 server 192.168.212.22, stratum 3, offset -0.062119, delay 0.02606 13 Jun 09:37:19 ntpdate[2368]: no server suitable for synchronization found user@workstation:~$ ntpdate -q 192.168.212.60 server 192.168.212.60, stratum 0, offset 0.000000, delay 0.00000 13 Jun 09:38:24 ntpdate[2373]: no server suitable for synchronization found user@workstation:~$ clockdiff -o 192.168.212.60 ...... host=192.168.212.60 rtt=177(265)ms/0ms delta=-17952260ms/-17952260ms Wed Jun 13 09:38:58 2018 Content of Sawyer's FSM: Hostname: 02170CP00029 IP type: Static IP IP Address*: 192.168.212.60 IP Netmask*: 255.255.252.0 IP Gateway*: 192.168.215.254 # That is a dummy one as there is no gateway configured on my network DNS: # Empty NTP Servers: 192.168.212.22 ROS Naming Type: ROS_IP Timezone: Europe/Paris Let me know if you need further information. Thanks Ian. Link to comment Share on other sites More sharing options...
Ian McMahon Posted June 13, 2018 Share Posted June 13, 2018 That is very helpful information. I think I now have a good handle on your network topology. Thank you Ulysse. After looking through your information, I realized it would be helpful to see your NTP Server's config file. Is there any chance you could supply that? Also, is this NTP server connected to the Internet or an absolute time device? Thank you, ~ Ian Link to comment Share on other sites More sharing options...
Ulysse Posted June 14, 2018 Author Share Posted June 14, 2018 I cannot have access to the NTP server but here is what I know about it: OS is CentOS Linux release 7.2 NTP version is 4.2.6p5 It uses the `time.my.company.fr` server as a reference, which is connected to the Internet Do you have specific informations you want me to ask for? Thanks Ian. Link to comment Share on other sites More sharing options...
David Khosid Posted August 31, 2018 Share Posted August 31, 2018 On 6/11/2018 at 10:04 AM, Ulysse said: How can I investigate on why is the robot not connecting to the server ? I gained some experience with making Sawyer + NTP work in corporate environment. Is your problem still relevant? In my case, I used tcpdump (Wireshark) to capture NTP traffic. It helped me to realize that Sawyer was silently rejecting new NTP configurations in situations when these configurations had accessibility limitations. The limitations of the new NTP configurations were not known to me and no indications were provided by Sawyer that it didn’t use the configured setup. I observed with Wireshark that even after updating "NTP Servers" on Sawyer, the robot continued to query NTP servers that were our OLD configuration (these old NTP servers were not available). It also became obvious that until the robot got response from new NTP servers, the setup presented to the user was not in sync with its internal configurations. A successful update happened after Sawyer was getting response from one of the NTP servers in the new setup. From the header of your /etc/ntp.conf I see that NTP policies are managed. I have some insights about this part which may be critical to success of setting your NTP server. What company are you from? :) You can message me in private. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.