Question: The "others" event callback

[cheton]

Hi @whimsycwd

this.controller.on('others', (res) => {
    this.emitAll('serialport:read', res.raw);

    moveOn(res, false);
});

With the above code, the moveOn() function is called within the "others" event callback. The "others" event will be emitted if the message was not able to parse (see https://github.com/whimsycwd/snapjs/blob/master/src/app/controllers/Marlin/Marlin.js#L349-L351).

May I know for what kinds of messages that need to handle in the "others" callback when running a G-code program? Thank you.

[whimsycwd]

About others

1. When you use M190 S60 to set temperature. Firmware will return 'ok /(T:[0-9.-]+).*(B:[0-9.-]+)/g;' then /(T:[0-9\.\-]+).*(B:[0-9\.\-]+)/g; periodically. Which can't be parsed.
2. I didn't enumerate all scenario before, use a other to catch response and move on to prevent machine stop... I don't remember why I did this. Bad habbit.

The way I handle this issue is problematic and buggy. It broke the protocol: Send and reply(ok).
I have a question haven't verified yet, I am not fully understand the stability of the USB cable. Will it lost data. What will it happen if we lost a 'Ok'? or Marlin failed to receive gcode from host software and didn't response. i.e. Do we need a headbeat in this simple send and reply protocol.

I am checking this project https://github.com/kliment/Printrun, They sent GCode with checksum to verify integrity of data, so I guess data lost is possible.

But I am little worry about this issue.

More About GCode sender

1. Current implementation is wrong, it will send more than receiving 'ok', I am working on fixing it,
2. I add a periodic query to get current position in parallel with standard gcode sender. instead of using one queue. This is wrong. cause code mess.
3. It will have issue if we use M109 S200 to set temperature. This command will take a long time to execute. Marlin is busy with executed command and received too much command to store in the buffer. Some of the gcode will be discarded.
4. Printrun sends GCode adding extra line number and checksum, so the host software can resend data if firmware return error info. But How the heartbeat works, I am still checking.

I feel sorry that I told you that I am busy with other staff so that I can't contribute back again and again.
I have written mess code, I have the ambition to clean up.
Let's start with GCode sender. This is a critical issue. It can be dangerous for CNC if it lost command. So I will solve it ASAP.

I think I can make it right in two weeks(at most 3 weeks). And migrate it base at your branch to contribute back.
Or you can finish before me. I can help you test it.

Let me know what you think.

Bests,
David

[cheton]

Thank you for the information. I'm currently working on Marlin firmware support using the Snapmaker. For the issue you mentioned above, I may have to hold program execution when receiving M109, M190, or similar commands that will take long time to execute. I will get back to you when I found the solution.

https://github.com/cncjs/cncjs/blob/master/src/app/controllers/Marlin/MarlinController.js

[cheton]

Documentation

• M190 Optionally set the target temperature for the heated bed, then wait in a loop until that target temperature is reached.
• M109 Optionally set the target temperature for an extruder, then wait in a loop until the target temperature is reached.
• M140 Set the target temperature for the heated bed and continue on.
• M104 Set the target temperature for a hotend and continue on.

According to the documentation, both M190 and M109 will wait for the temperature to be reached before proceeding with the next command. If users do not want to pause for heating, they have to use M140 and M104 instead.

In CNCjs 1.9.11 (https://github.com/cncjs/cncjs/releases/tag/v1.9.11), I added a new feature that can handle M0, M1, M2, M30, and M6 tool change commands, and pause program execution. The web UI will show a notification message or a modal window asking user to continue:

https://github.com/cncjs/cncjs/blob/master/src/app/controllers/Marlin/MarlinController.js#L345-L369

Notification message

Modal window

For M109 and M190 commands, I can use the same mechanism to pause program execution, and show a modal window to display a temperature chart with current temperature. During the waiting period, user can force stopping with M112, or wait until the target temperature has reached.

That's what I may want to do next. Please let me know if I missed anything.

[cheton]

There is one thing I need to clarify. The G20 (inch units) command does not work as expected when I execute the following:

G20
G0 X1

Actually, it moved by 1mm, not 1 inch. The behavior seems incorrect according to the documentation: http://marlinfw.org/docs/gcode/G020.html

[cheton]

M109 and M190 will block execution if temperature is not set by the S or R parameter.
MarlinFirmware/Marlin#6308
PR: MarlinFirmware/Marlin#6311

[cheton]

Hi @whimsycwd

From my test result, Marlin will respond an "ok" until the target temperature is reached, as shown below:

Set heated bed temperature

M190 S60
 T:25.2 /0.0 B:25.5 /60.0 B@:0 @:0 W:?
 T:25.2 /0.0 B:25.6 /60.0 B@:127 @:0 W:?
 T:25.4 /0.0 B:25.8 /60.0 B@:127 @:0 W:?
 T:25.3 /0.0 B:26.3 /60.0 B@:127 @:0 W:?
 T:25.3 /0.0 B:26.7 /60.0 B@:127 @:0 W:?
 T:25.4 /0.0 B:27.0 /60.0 B@:127 @:0 W:?
 T:25.5 /0.0 B:27.3 /60.0 B@:127 @:0 W:?
 T:25.2 /0.0 B:27.7 /60.0 B@:127 @:0 W:?
   :   :   :
 T:25.7 /0.0 B:59.7 /60.0 B@:127 @:0 W:6
 T:25.6 /0.0 B:59.8 /60.0 B@:127 @:0 W:5
 T:25.6 /0.0 B:59.9 /60.0 B@:127 @:0 W:4
 T:25.7 /0.0 B:60.1 /60.0 B@:0 @:0 W:3
 T:25.6 /0.0 B:60.0 /60.0 B@:0 @:0 W:2
 T:25.5 /0.0 B:59.8 /60.0 B@:0 @:0 W:1
 T:25.7 /0.0 B:59.7 /60.0 B@:0 @:0 W:0
ok

Set extruder temperature

M109 S60
 T:26.5 /60.0 B:60.4 /60.0 B@:127 @:0 W:?
 T:26.4 /60.0 B:60.5 /60.0 B@:127 @:127 W:?
 T:26.9 /60.0 B:60.6 /60.0 B@:127 @:127 W:?
 T:27.6 /60.0 B:60.7 /60.0 B@:0 @:127 W:?
 T:28.8 /60.0 B:60.5 /60.0 B@:0 @:127 W:?
 T:30.0 /60.0 B:60.5 /60.0 B@:0 @:127 W:?
    :    :    :
 T:61.6 /60.0 B:59.5 /60.0 B@:127 @:8 W:6
 T:61.2 /60.0 B:59.7 /60.0 B@:127 @:11 W:5
 T:60.9 /60.0 B:59.8 /60.0 B@:127 @:14 W:4
 T:60.7 /60.0 B:59.9 /60.0 B@:127 @:15 W:3
 T:60.5 /60.0 B:60.1 /60.0 B@:127 @:16 W:2
 T:60.3 /60.0 B:60.2 /60.0 B@:127 @:15 W:1
 T:60.2 /60.0 B:60.4 /60.0 B@:127 @:12 W:0
ok

I think it won't break the send-response protocol if we only care about the "ok" response. Also, the query position timer should be disabled during the period of waiting for temperature with M109 or M190 command.

[cheton]

A refined temperature parser might look below, it should be able to parse the response of M105, M109, M190. See Marlin/Marlin_main.cpp for details.

const parse = (line) => {
    const payload = {};

    let r = line.match(/^(?:ok)?\s*(?:(?:(T|B|T\d+):([0-9\.\-]+)\s+\/([0-9\.\-]+)(?:\s+\((?:[0-9\.\-]+)\))?)|(?:(@|B@|@\d+):([0-9\.\-]+))|(?:(W):(\?|[0-9]+)))/i);
    if (!r) {
        return null;
    }

    const re = /(?:(?:(T|B|T\d+):([0-9\.\-]+)\s+\/([0-9\.\-]+)(?:\s+\((?:[0-9\.\-]+)\))?)|(?:(@|B@|@\d+):([0-9\.\-]+))|(?:(W):(\?|[0-9]+)))/ig;

    while((r = re.exec(line))) {
        if (r[1]) { // T:293.0 /0.0
            const key = r[1];
            const current = Number(r[2]) || 0;
            const target = Number(r[3]) || 0;
            payload[key] = [current, target];
        } else if (r[4]) { // @:0 or @B:0
            const key = r[4];
            const value = Number(r[5]) || 0;
            payload[key] = value;
        } else if (r[6]) { // W:? or W:9
            const key = r[6];
            const value = (r[7] === '?') ? r[7] : Number(r[7]);
            payload[key] = value;
        }
    }

    console.log(payload);
};

M105

parse('ok T:293.0 /0.0 B:25.9 /0.0 @:0 B@:0');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], '@': 0, 'B@': 0 }

parse('ok T:293.0 /0.0 B:25.9 /0.0 T0:293.0 /0.0 T1:100.0 /0.0 @:0 B@:0 @0:0 @1:0');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], T0: [ 293, 0 ], T1: [ 100, 0 ], '@': 0, 'B@': 0, '@0': 0, '@1': 0 }

parse('ok T:293.0 /0.0 (0.0) B:25.9 /0.0 T0:293.0 /0.0 (0.0) T1:100.0 /0.0 (0.0) @:0 B@:0 @0:0 @1:0');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], T0: [ 293, 0 ], T1: [ 100, 0 ], '@': 0, 'B@': 0, '@0': 0, '@1': 0 }

parse('ok T:293.0 /0.0 (0.0) B:25.9 /0.0 T0:293.0 /0.0 (0.0) T1:100.0 /0.0 (0.0) @:0 B@:0 @0:0 @1:0 W:?');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], T0: [ 293, 0 ], T1: [ 100, 0 ], '@': 0, 'B@': 0, '@0': 0, '@1': 0, W: '?' }

parse('ok T:293.0 /0.0 (0.0) B:25.9 /0.0 T0:293.0 /0.0 (0.0) T1:100.0 /0.0 (0.0) @:0 B@:0 @0:0 @1:0 W:0');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], T0: [ 293, 0 ], T1: [ 100, 0 ], '@': 0, 'B@': 0, '@0': 0, '@1': 0, W: 0 }

M109 / M190

parse(' T:293.0 /0.0 B:25.9 /0.0 @:0 B@:0');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], '@': 0, 'B@': 0 }

parse(' T:293.0 /0.0 B:25.9 /0.0 T0:293.0 /0.0 T1:100.0 /0.0 @:0 B@:0 @0:0 @1:0');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], T0: [ 293, 0 ], T1: [ 100, 0 ], '@': 0, 'B@': 0, '@0': 0, '@1': 0 }

parse(' T:293.0 /0.0 (0.0) B:25.9 /0.0 T0:293.0 /0.0 (0.0) T1:100.0 /0.0 (0.0) @:0 B@:0 @0:0 @1:0');
// { T: [ 293, 0 ], B: [ 25.9, 0 ], T0: [ 293, 0 ], T1: [ 100, 0 ], '@': 0, 'B@': 0, '@0': 0, '@1': 0 }

[cheton]

I made it works with my master branch, you can see my commits at
https://github.com/cncjs/cncjs/commits/master

Source Code
https://github.com/cncjs/cncjs/tree/master/src/app/controllers/Marlin
https://github.com/cncjs/cncjs/tree/master/src/web/widgets/Marlin

Controller State
http://localhost:8000/api/controllers

{
    port: "/dev/tty.wchusbserialfa130",
    baudrate: 115200,
    sockets: [
        "YF7zNKTPtZ_H7DEPAAAB"
    ],
    ready: true,
    controller: {
        type: "Marlin",
        settings: {
            firmwareName: "Marlin 1.1.0",
            protocolVersion: "1.0",
            machineType: "3D",
            extruderCount: 1,
            uuid: "cede2a2f-41a2-4748-9b12-c55c62f367ff"
        },
        state: {
            pos: {
                x: "0.00",
                y: "0.00",
                z: "0.00",
                e: "0.00"
            },
            modal: {
                motion: "G0",
                wcs: "G54",
                plane: "G17",
                units: "G21",
                distance: "G90",
                feedrate: "G94",
                program: "M0",
                spindle: "M5",
                coolant: "M9"
            },
            ovF: 100,
            ovS: 100,
            heater: {
                extruder: {
                    deg: "41.4",
                    degTarget: "40.0",
                    power: "2"
                },
                bed: {
                    deg: "25.2",
                    degTarget: "0.0",
                    power: "0"
                },
                wait: "0"
            },
            rapidFeedrate: 0,
            feedrate: 0,
            spindle: 0
        }
    },
    feeder: {
        hold: false,
        holdReason: null,
        queue: 0,
        pending: false,
        changed: false
    },
    sender: {
        sp: 0,
        hold: true,
        holdReason: {
            data: "M109"
        },
        name: "jsdc-snapmaker.nc",
        context: {
            xmin: 0,
            xmax: 0,
            ymin: 0,
            ymax: 0,
            zmin: 0,
            zmax: 0,
            posx: 0,
            posy: 0,
            posz: 0,
            pose: 0,
            modal: {
                motion: "G0",
                wcs: "G54",
                plane: "G17",
                units: "G21",
                distance: "G90",
                feedrate: "G94",
                program: "M0",
                spindle: "M5",
                coolant: "M9"
            }
        },
        size: 21284,
        total: 927,
        sent: 1,
        received: 1,
        startTime: 1510589200696,
        finishTime: 0,
        elapsedTime: 48293,
        remainingTime: 44719318
    },
    workflow: {
        state: "running"
    }
}

[whimsycwd]

1. GCode G28 X0 Y0 will return X:-7.00 Y:-6.00 Z:0.00 E:0.00 Count X: -2800 Y:-2400 Z:0
   This will be parse as position query.
   And position query's ok will be marked then ignore.

2. M114 will return 'ok ...'
   If gcode include this command. Printing will stop.

M105
G28 X0 Y0
G1 X20 Y20

cncjs/cncjs#228

Still checking https://github.com/kliment/Printrun/blob/master/printrun/printcore.py, do we need line number and checksum.. resend mechanism?

[cheton]

A preview of the Marlin widget

[whimsycwd]

Hi, cheton,
我英语不是很好， 写起来会比较麻烦一点， 我速记一些中文草稿， 就不翻译成英文了。

我特别喜欢你加的新功能， 温度加热的提示框以及加热是那个闪烁得小图标很赞!

我把你代码拉下来来测试一下， 尝试发送3D 打印的GCode， 还是会有问题, GCode 发送会被截断
测试的Case
这是由于温度的加热会需要很长的时间
https://github.com/cncjs/cncjs/blob/master/src/app/controllers/Marlin/MarlinController.js#L618-L626
这个发送逻辑可能是有问题的, Marlin 的接收处对于多出来的命令指令会采取覆盖和截断的处理。 比如G0 X10 Y10 Z10会被截为 G0 X10 Y10.

---

8f8470f

Background

关于要不要做行校验的问题， 我和我们的固件工程师讨论交流之后得出了以下结论。
USB 转串口， 协议很复杂， 本来就做了校验， 所以不会有问题。 CH340 -> 单片机链路可能出问题， 不过出问题的可能性非常低。 如果出现多半是下位机固件的问题。 因此决定不实现类似Printrun 的校验机制了。 我修正了GCode 发送的处理逻辑， 通过一个锁的方式来使命令串行发送

Requirement

1. 严格遵守 request/reply 协议， 没有收到reply 不能够再继续发命令
   • Marlin 的缓冲区是有固定的大小的（512Byte)。 如果不保持接收一条发一条的速度, 那么缓冲区可能被溢出， 会出现的结果就是， 命令被截断。
2. 三个输入源
   • 1. feeder(导入GCode), 2. sender(GCode 输入), 3. 计时器定时查阅温度和状态
   • Sender 优先级高于feeder, 当在发送GCode 的时候， feeder 会被starve, 也就是在执行GCode 时候不会执行feeder.
   • 不希望计时器的位置查询和温度查询被starve

Design

1. lastCmdType // null(无指令) sender, feeder, timer, 用来标示发出的命令类型
2. lastCmd // 记录上一次发出的指令, 方便打出日志排查
3. timerFlag // null(没有时间标记), position, temperature
4. cmdCount // Sender + Feeder 命令累加器.

发出指令

1. gcode:start
2. 指令接收处 'ok', 得到数据的回应折后发起一条新的指令
3. Timer 处， 定时出发， 当lastCmdType = null(没有在等待回馈指令)并且feeder 和 Sender 都为空的时候Kick 第一个定时查询.
4. gcode， 把指令导入 feeder 的时候, 在判断发出之前, 当lastCmdType = null(没有在等待回馈指令), sender 发送完毕的时候
5. 命令行控制台的指令是绕过feeder 直接写的。这个应该被禁止 @todo
   • @cheton 通过Feeder 来写入用户数据是设计的意图吧?
   • 不过Marlin 的缓冲区与 512Byte, 应该也遇不到问题?

总结

1. lastCmdType 是一个资源锁, 必须获得这个资源锁， 才能开始Kick一个发出指令。 从而保证feeder & sender & Timer 的串行发指令
2. 通过 cmdCount 来来累加Sender + Feeder 的发送指令数， 每发送十个指令来发送以恶个Timer 指令。
3. Timer 本身在获取不到锁的情况下不发起指令， 只标记TimerFlag, 由接收处决定要不要插入Timer 的指令(Position & Temperature).

[cheton]

https://github.com/cncjs/cncjs/blob/master/src/app/controllers/Marlin/MarlinController.js#L618-L626
这个发送逻辑可能是有问题的, Marlin 的接收处对于多出来的命令指令会采取覆盖和截断的处理。 比如G0 X10 Y10 Z10会被截为 G0 X10 Y10.

1. 這段程式碼從 Grbl controller 抄過來的，主要是針對 Grbl 做的一些處理，如果在 bootstrap 後沒反應或因某些原因卡住，可以透過這方式讓 position query 回復正常。這部分我想在 Marlin 應該是不需要，我先嘗試拿掉看看，再做些更完整的測試。

2. 資源鎖這 idea 很棒，我會再找時間研究一下如何整理目前的 code。

3. 命令行控制台 (Console widget) 的確不應該繞過 feeder 寫入，這部分未來會修正。

P.S. 最近要照顧家裡的兩個小孩，常常時間不太夠用，所以有時候會晚些才回覆 :P

[cheton]

感謝你的幫忙，經過這幾天的修改後應該可以動了，目前是用你給的 G-code sample 做測試。

我簡要說明一下這邊的修改

1. MarlinController.js 新增一個 history object，writeSource & writeLine 相當於 lastCmdType & lastCmd。

   history = {
       writeSource: '',
       writeLine: ''
   };

   Write Source
   Write Source 分成四種類型，其中 'client' 是透過 controller.write() 寫進來的。

   https://github.com/cncjs/cncjs/blob/master/src/app/controllers/Marlin/constants.js

   export const WRITE_SOURCE_CLIENT = 'client';
   export const WRITE_SOURCE_FEEDER = 'feeder';
   export const WRITE_SOURCE_SENDER = 'sender';
   export const WRITE_SOURCE_QUERY = 'query';

   SerialConnection.js
   v2 (edge branch) 計畫要支援 TCP Socket Connection，所以在 master branch 已經先把 SerialConnection 獨立出來
   https://github.com/cncjs/cncjs/blob/master/src/app/lib/SerialConnection.js
   https://github.com/cncjs/cncjs/blob/edge/src/app/lib/SocketConnection.js

   writeFilter
   writeFilter 是資料寫出前的最後關卡， 在這裡去更新 writeSource 和 writeLine 可以確保所有更改都集中在同一處。

   writeFilter: (data, context) => {
       const { source = null } = { ...context };
       const line = data.trim();
   
       this.history.writeSource = source; // 'client', 'feeder', 'sender', 'query'
       this.history.writeLine = line;
   
       // Update modal state
           :    :    :
   }

2. 調換了 temperature 和 ok 事件的順序，讓 temperature 優先於 ok，這樣 ok T:27.0 /0.0 B:26.8 /0.0 B@:0 @:0 就能正常輸出到 Console widget，不會因為 writeSource 被清掉而無法判讀寫入來源。

   Marlin.js

   // The 'ok' event (w/ empty response) should follow the 'temperature' event
   this.emit('temperature', payload);
   
   // > M105
   // < ok T:27.0 /0.0 B:26.8 /0.0 B@:0 @:0
   if (payload.ok) {
       // Emit an 'ok' event with empty response
       this.emit('ok');
   }

   MarlinController.js

   this.controller.on('temperature', (res) => {
       if (_.includes(['client', 'feeder'], this.history.writeSource)) {
           this.emit('serialport:read', res.raw);
       }
   });

   this.controller.on('ok', (res) => {
       if (res) {
           if (_.includes(['client', 'feeder'], this.history.writeSource)) {
               this.emit('serialport:read', res.raw);
           } else if (!this.history.writeSource) {
               this.emit('serialport:read', res.raw);
               log.error('"history.writeSource" should not be empty');
           }
       }
   
       this.history.writeSource = null;
       this.history.writeLine = '';
          :   :   :
   });

3. 要如何避免 starvation，我最後是選擇用時間代替計數器，如果超過 2000ms 都沒更新，就觸發下一次的 query。

   // Perform preemptive query to prevent starvation
   const now = new Date().getTime();
   const timespan = Math.abs(now - this.query.lastQueryTime);
   if (this.query.type && timespan > 2000) {
       this.query.issue();
       return;
   }

   以下是 query object 的結構，利用一個 lastQueryTime 紀錄上次 query 的時間

   query = {
       // state
       type: null, // 'position', 'temperature'
       lastQueryTime: 0,
   
       // action
       issue: () => {
           if (!this.query.type) {
               return;
           }
   
           const now = new Date().getTime();
   
           if (this.query.type === QUERY_TYPE_POSITION) {
               this.connection.write('M114\n', {
                   source: WRITE_SOURCE_QUERY
               });
               this.lastQueryTime = now;
           } else if (this.query.type === QUERY_TYPE_TEMPERATURE) {
               this.connection.write('M105\n', {
                   source: WRITE_SOURCE_QUERY
               });
               this.lastQueryTime = now;
           } else {
               log.error('Unsupported query type:', this.query.type);
           }
   
           this.query.type = null;
       }
   };