DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Comments
Claims 1 and 10 have been amended by applicant. Claim 19 is newly added by applicant.
Claims 1-5, 9-14, and 18-19 remain rejected.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/20/2025 and 03/05/2026 were filed after the mailing date of the Non-Final Rejection on 09/19/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claim 10 is objected to because of the following informalities:
In claim 10, line 15, “estimation model” should read, --depth estimation model--.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 9-14, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wei (U.S. Patent No. US10,953,281B2), in view of Lim (Korean Patent Publication No. KR102055146B1), Brown (International Patent Publication No. WO2020250046A1), and Nguyen (PG Patent Publication No. US2022/0355179A1).
Regarding claim 1, Wei shows a treadmill (Wei, treadmill M, col. 3, line 52), comprising: a treadmill body (Wei, “a base 10, a running track 12 and a detection module 14”, col. 3, lines 53-54), comprising a running belt (Wei, running track 12); an event-based vision sensor (Wei, “the detection module 14 may include a light emitter 140 and a light sensor 142”, col. 4, lines 1-2; The light sensor of Wei shows the event-based vision sensor of the claimed invention), disposed on the treadmill body (Wei, See annotated FIG. 1 below) and generating a sensing image, wherein each pixel in the sensing image generated by the event-based vision sensor represents variation of light intensity (Wei, “the light sensor 140 may be an image sensor, such as CMOS or CCD, or may include a plurality of photodiodes. Specifically, the light sensor 140 is configured to obtain a captured image including at least one object image indicating position of the user A… When an incident light travels to a part of the body of the user A, such as legs or shoes, the incident light will be scattered, diffused, and reflected, and the light sensor 140 may detect image by receiving the scattered light, diffused light, and reflected light and obtain the captured image… Specifically, the object image could be a dot or a group of strips depends on the type of light emitted from the light emitter 142. In addition, the feature of the at least one object image includes at least one of a size and a variation frequency of a pattern”, col. 4, lines 10-36; It is well known in the art that CMOS and CCD sensors are designed to detect and measure changes in light intensity. More specifically, photodiodes are designed to detect and measure light intensity); a processor (Wei, control unit 106, col. 4, line 42), coupled to the event-based vision sensor (Wei, “the treadmill M may further include a control unit 106 coupled to the detection module 14 and configured to adjust the speed of the running of the running track according to the position or the speed of the user”, col. 4, lines 41-45), obtaining the sensing image, and performing runner detection on the sensing image, in response to determining that a runner is detected from the sensing image, the processor inputting the sensing image to a depth estimation model, obtaining a depth map output by the depth estimation model and obtaining position information of the runner relative to the running belt, according to the depth map, and controlling a running speed of the running belt according to the position information of the runner (Wei, “the treadmill M may further include a control unit 106 coupled to the detection module 14 and configured to adjust the speed of the running of the running track according to the position or the speed of the user. Specifically, the light sensor may include one or more processors or microcontrollers for processing and identifying the object image in the captured image. When the light sensor 140 identifies the at least one object image in the captured image, the light sensor 140 generates information related to the object images for the control unit 106 to adjust the speed of the running track 12… the control unit 106 may also include an image-processing unit for receiving the captured image and calculating the characteristic properties of the object image according to the captured image. The control unit 106 may further be configured to adjust the speed of the running track 12 according to the characteristic properties of the object image accordingly… For example, when the light sensor 140 detects that the user A is located at the near zone NZ, the control unit 106 may be configured to accelerate the running track 12; when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”, col. 4, lines 47-col. 5, line 28; The depth estimation model is shown by the detection of the user within a specific zone of Wei with the image-processing unit as the specification of the claimed invention describes the depth estimation model, in paragraph 0027, “the processor 130 inputs the sensing image to the depth estimation model to obtain depth information of the runner, thereby obtaining the position information of the runner on the running belt 112”. See FIG. 1 below. The calculation of characteristic properties, such as the position of the user, from the captured image of Wei teaches the depth estimation map of the claimed invention as the specification of the claimed invention discloses in paragraph 0033, “The processor 130 may input the sensing image Img_1 to a depth estimation model M1, and obtain a depth map D1 output by the depth estimation model M1. The depth map D1 may include a depth value corresponding to each pixel in the sensing image Img_1, that is, multiple depth values in the depth map D1 respectively correspond to multiple pixels in the sensing image Img_1”).
PNG
media_image1.png
579
406
media_image1.png
Greyscale
Wei
Wei fails to show, the event-based vision sensor comprises a dynamic vision sensor; the depth estimation model comprising a monocular depth estimation model trained using a deep learning model; and specifically in response to detection of a moving object, the processor being configured to input the sensing image including the moving object to the depth estimation model comprising the monocular depth estimation model, obtain the depth map output by the depth estimation model, obtain position information of the moving object, and determine a second distance between the moving object and a reference position according to the depth map, so as to control the running speed of the running belt according to the position information of the moving object.
However, Lim, from the same field of endeavor, discloses a method for checking out sports motion using event-based vision sensor and apparatus for the same and teaches a dynamic vision sensor (Lim, “The event-based vision sensor used in an embodiment of the present invention is an apparatus for generating at least one event signal by detecting a part of an object in which a motion is generated. For example, a dynamic vision sensor is a commercially available event-based sensor. Instead of transmitting a frame sequence, which is a continuous signal of a still image, a dynamic vision sensor detects a change in light intensity to identify a specific scene in the scene when an event occurs. Only a change in the luminance (e.g., event) of the pixel at the position is transmitted. In other words, the output of the dynamic vision sensor is a stream of events where each event is associated with a particular state”, page 3).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have replaced the detection module which includes a light emitter and a light sensor of Wei with the dynamic vision sensor of Lim as Lim discloses in page 3, “the event signal output from the dynamic vision sensor carries only the information of the pixel detected by the user's movement, it can analyze the image much faster and more accurately than the conventional camera transmitting thousands of frames per second”. In light of this modification, the event signal output of Lim teaches the sensing image of the claimed invention. Lim also discloses on page 3, “Conventional vision sensors collect the scene as a sequence of frames from the moving image and include all photographic elements including pixels within one frame. In the frame sequence, consecutive frames before and after are collected in units of time, and there is overlapped pixel information without change. As such, storing and processing redundant pixel information frame by frame wastes storage space, processing time, and battery power” which disclose the function of the preferred light sensors, CMOS and CCD sensors, of Wei, making this modification obvious. Moreover, dynamic vision sensors generate a sensing image, wherein each pixel in the sensing image generated by the event-based vision sensor represents variation of light intensity, thereby teaching the limitation of the claimed invention “generating a sensing image, wherein each pixel in the sensing image generated by the event-based vision sensor represents variation of light intensity” with this modification.
Furthermore, Brown, a teaching reference showing a similar problem of estimating the position of a user with a capture device, discloses a method and system for monocular depth estimation of persons and teaches a monocular depth estimation model (Brown, “the human joint 3D estimation system 110 may comprise a capture device 120 to capture a digital image, a 2D skeleton detector 130, which may detect and crop humans in an image and localize human joint positions in the image, a 2D joint heatmap generator 140, which may generate a positional heatmap for each kind of joint to be localized, a depth heatmap estimator 150, which may generate 1 D depth heatmaps for each kind of localized joint, and a 3d joint constructor 160, which may select a depth value for each kind of joint from the depth heatmaps and combine this information with the 2D joint positions to produce 3D joint positions”, paragraph 0017; The human joint 3D estimation system of Brown teaches the monocular depth estimation model of the claimed invention) trained using a deep learning model (Brown, “A depth heatmap estimator may be embodied by a trained machine learning model. This model may be a convolutional neural network (CNN), or alternatively a RNN, random forest, or deep neural network”, paragraph 0026).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have replaced the image-processing unit of Wei that detects the user within a specific zone of Wei with the human joint estimation system of Brown as Brown discloses in paragraphs 0003-0005, “Depth sensing hardware is often expensive and limiting in its use, such as having a limited range and may only be used inside… existing approaches often exhibit limited accuracy and robustness, especially in challenging poses, environments and conditions. Thus, there exists the need for a better approach to monocular RGB estimation of human joint depths”. Brown also discloses in paragraph 0018, “A capture device 120 may represent any device or method for acquiring a digital image or video frame”, and dynamic vision sensors are devices for acquiring digital visual data, making this modification obvious. Moreover, Brown discloses in paragraph 0037-0040, “The 3D joint constructor 160 may accept 2D joint positions for one detected person, for example from the 2D skeleton detector 130, and corresponding joint depth heatmaps, produced by the depth heatmap estimator 150, and output 3D joint positions… By combining the 2D joint heatmap with the depth estimation for each kind of joint, a 3D location for each joint can be estimated. A rendering 740 of a skeleton based on these 3D locations is shown”. Therefore, Wei, in view of Brown teaches the limitation of the claimed invention “the processor obtaining a depth map output by the depth estimation model and obtaining position information of the runner relative to the running belt” with the 3D locations for each joint of Brown.
However, Nguyen, from the same field of endeavor, teaches in response to detection of a moving object, determine a second distance between the moving object and a reference position (Nguyen, “the module 202 can receive distance information from an object, and determine the object is within a certain area or proximity of the exercise machine. Further, using multiple detected locations, the module 202 can determine if the object is moving towards the machine and/or acceleration/decelerating with respect to the machine”, paragraph 0039; The distance information from the moving object of Nguyen teaches the second distance between the moving object and a reference position) so as to control the running speed of the running belt according to the position information of the moving object (Nguyen, “In some embodiments, the operation modification module 204 is configured and/or programmed to modify a current operation of the exercise machine based on detecting the object is proximate to the exercise machine… the module 204 can perform an action to control a mechanical operation of the exercise machine, such as by transmitting instructions to a controller 220 or control system of the exercise machine. For example, when the exercise machine is a treadmill, the operation modification module 204 causes the controller 220 to slow or stop the movement of the running surface of the treadmill”, paragraphs 0041-0042).
Nguyen also discloses in paragraph 0040, “Of course, the module 202 can utilize various combinations of sensors and captured information, as described in greater detail herein. For example, the object detection module 202 can (1) receive information from a time-of-flight sensor that detects the object is within the area surrounding the exercise machine and (2) a computer vision (CV) sensor that determines the object detected to be within the area surrounding the exercise machine is moving towards the exercise machine and/or a millimeter wave (mmWave) sensor that determines the object detected to be within the area surrounding the exercise machine is moving towards the exercise machine”. The time-of-flight sensors of Nguyen are comparable to the event-based vision sensor of Lim such that both sensors detect motion. The OpenCV techniques of Nguyen disclosed in paragraphs 0031 and 0032 of Nguyen are used for depth estimation techniques, as well as, millimeter wave sensors of Nguyen disclosed in paragraphs 0033-0035. Therefore, the module of Nguyen is comparable to the human joint 3D estimation system of Brown such that both are models for depth estimation based on data collected by sensors that detect motion. More specifically, as seen in paragraph 0033 of Nguyen, “The mmWave sensors utilize a Frequency Modulated Continuous Wave (FMCW) mechanism that transmits and receives chirps (e.g., waves at 60 Ghz or 24 Ghz) and can capture three dimensions of information (e.g., velocity, range or distance, and angle) for an object proximate to the exercise machine”, the three dimensions of information, including the distance for an object proximate to the machine, is comparable to the depth map of the claimed invention.
PNG
media_image2.png
445
476
media_image2.png
Greyscale
Nguyen
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have the modified treadmill of Wei also control the speed of a running track upon the detection of a moving object according to the methods already disclosed by Wei, the dynamic vision sensor of Lim, and the human joint 3D estimation system of Brown to further “provide a durable and safe treadmill” (Nguyen, col. 1, lines 53-54). Wei already discloses controlling the speed of a running track of a treadmill based on the speed or position of the runner/user. Nguyen simply improves on this by controlling the speed of the running surface of the treadmill if a moving object is heading towards the treadmill. Both Brown and Nguyen disclose utilizing sensors for detecting motion to determine distances through comparable processes, making this modification obvious. More specifically, it would have been obvious for the processor, as shown by Wei, to be configured to input the sensing image including the moving object, taught by Nguyen, to the human joint 3D estimation system, obtain the depth map output by the human joint 3D estimation system, obtain position information of the moving object, as Wei already shows this process for detecting the position of a runner on the belt. This modification simply utilizes the combination sensors of Nguyen to process, in the same manner as that of the runner, for detecting the position of a moving object near the treadmill.
Regarding claim 2, Wei, in view of Lim, Brown, and Nguyen, teaches the treadmill according to claim 1, wherein the processor performs person detection on the sensing image, obtains a person bounding box on the sensing image, and determines whether the runner is detected according to whether the person bounding box is located within a predetermined area on the sensing image (Wei, “As shown in FIG. 3, the running zone 103 may be divided into several zones, and when the light sensor 140 identifies which zone the user A is located, the control unit 106 may control the speed of the running track 12 directly. For example, the running track 12 may be divided into three zones, near zone NZ, middle zone MZ, and far zone FZ with respect to positions on the running zone 103 along a running direction D1 of the user. It should be noted that the first row R1 and the neighboring rows correspond to a region of the running track 12 that is near the front of the treadmill M in the example of the 1D image and 2D image, such as the near zone NZ in FIG. 3. In other embodiment, the three zones could only correspond to particular rows in the captured image, such as rows R1-R3 corresponding to near zone NZ, rows R4-R6 corresponding to middle zone MZ, and rows R7-R9 corresponding to far zone FZ”, col. 4, lines 65-67, col. 5, lines 1-16; In the broadest reasonable interpretation of the claim, the running zone 103 disclosed by Wei, as seen in FIGS. 2A and 3 below, shows at least one bounding box of the claimed invention as it is well known in the art that in image processing, a bounding box is a rectangular-shape that defines the location and size of an object in an image).
PNG
media_image3.png
313
177
media_image3.png
Greyscale
PNG
media_image4.png
623
371
media_image4.png
Greyscale
Wei
Regarding claim 3, Wei, in view of Lim, Brown, and Nguyen, teaches the treadmill according to claim 2, wherein if the person bounding box is not located within the predetermined area on the sensing image, the processor determines that the runner is not detected (Wei, “On the other hand, if the light sensor 106 detects that the user is located at one or both of the two rest zones 101 and 102, the control unit 106 may be configured to stop the running track 12”, col. 5, lines 28-31; When the user is in not in the running zone, i.e. either of the rest zones, and the treadmill stops the track, Wei therefore does not detect a user in the predetermined area of the running zone); and if the person bounding box is located within the predetermined area on the sensing image, the processor determines that the runner is detected (Wei, “The present embodiment further provides several cases for the control unit 106 to adjust the speed of the running track 12 according to the position of the user A. For example, when the light sensor 140 detects that the user A is located at the near zone NZ, the control unit 106 may be configured to accelerate the running track 12; when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”, col. 5, lines 17-27; Wei shows the location of a user in the running zone and the adjustment of the speed shows detection of the user).
Regarding claim 4, Wei, in view of Lim, Brown, and Nguyen, teaches the treadmill according to claim 1, wherein in response to determining that a runner is detected from the sensing image, the processor inputs the sensing image to the depth estimation model, obtains the depth map output by the depth estimation model (Wei, “In more detail, the light sensor 140 obtains a captured image including at least one object image indicating position of the user A, and the captured image could be formed as a 2D image or an 1D image, as shown in FIGS. 2A and 2B”, col. 4, lines 37-41; The at least one object image of Wei shows the runner being detected from the sensing image of the claimed invention. The modified treadmill of Wei teaches the processor inputting the sensing image to the dynamic vision sensors and obtaining the 3D locations for each joint with the human joint 3D estimation system of Brown), and determines a first distance between the runner and the reference position according to the depth map (Wei, “As shown in FIG. 3, the running zone 103 may be divided into several zones, and when the light sensor 140 identifies which zone the user A is located, the control unit 106 may control the speed of the running track 12 directly. For example, the running track 12 may be divided into three zones, near zone NZ, middle zone MZ, and far zone FZ with respect to positions on the running zone 103 along a running direction D1 of the user. It should be noted that the first row R1 and the neighboring rows correspond to a region of the running track 12 that is near the front of the treadmill M in the example of the 1D image and 2D image, such as the near zone NZ in FIG. 3. In other embodiment, the three zones could only correspond to particular rows in the captured image, such as rows R1-R3 corresponding to near zone NZ, rows R4-R6 corresponding to middle zone MZ, and rows R7-R9 corresponding to far zone FZ”, col. 4, line 67-col. 5, lines 1-16; The first distance between the runner and the reference position is shown by Wei with the user in a respective zone of Wei).
Regarding claim 5, Wei, in view of Lim, Brown, and Nguyen, teaches the treadmill according to claim 4, wherein if the first distance is greater than a first threshold value, the processor controls the running speed of the running belt to decrease (Wei, “When the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”, col. 5, lines 22-28; The threshold value of the claimed invention is shown by the middle zone of Wei); and if the first distance is not greater than the first threshold value, the processor maintains the running speed of the running belt (Wei, “when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12”, col. 5, lines 22-25).
Regarding claim 9, Wei, in view of Lim, Brown, and Nguyen, teaches the treadmill according to claim 1.
Wei fails to explicitly teach if the second distance is less than a second threshold value, the processor controls the running speed of the running belt to decrease; and if the second distance is not less than the second threshold value, the processor maintains the running speed of the running belt.
However, Nguyen teaches, if the second distance is less than a second threshold value, the processor controls the running speed of the running belt to decrease (Nguyen, “In some cases, a monitored region can include multiple sub-regions, and the treadmill 100 can perform different actions that are based on the detection of objects within one or more of the sub-regions… a closer sub-region 140 can be associated with a slowing of the treadmill 100”, paragraph 0023); and if the second distance is not less than the second threshold value, the processor maintains the running speed of the running belt (Nguyen, “if a region has three sub-regions (as depicted in FIG. 1B), the sub-regions can each be associated with a different action. For example, a sub-region 150 farthest from the treadmill 100 can be associated with a warning presented to the runner 105”, paragraph 0023; The sub-region of Nguyen which only outputs a warning teaches the maintaining of the running speed of the running belt as the closer sub-region and the inner sub-region, also disclosed in paragraph 0023, require the slowing or stopping of the treadmill. See FIG. 1B below).
PNG
media_image5.png
294
181
media_image5.png
Greyscale
Nguyen
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified treadmill of Wei to specifically adjust the speed of the treadmill dependent on the position of the moving object, taught by Nguyen, as Wei already discloses variable speed adjustment based on position of the user. Specifically, in col. 5, lines 19-28, Wei discloses “For example, when the light sensor 140 detects that the user A is located at the near zone NZ, the control unit 106 may be configured to accelerate the running track 12; when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”. Therefore, it would be obvious to apply the same variable speed adjustment for the treadmill based on the position of the moving object as well.
Regarding claim 19, Wei, in view of Lim, Brown, and Nguyen, teaches the treadmill according to claim 1, wherein in response to determining that the runner is detected from the sensing image, the processor is configured to perform motion detection on a background area in the sensing image to detect the moving object in the background area, and the background area is an area outside a person bounding box in the sensing image (As explained in claim 2 above, in the broadest reasonable interpretation of the claim, the running zone 103 disclosed by Wei, as seen in FIGS. 2A and 3 above, shows a person bounding box of the claimed invention as it is well known in the art that in image processing, a bounding box is a rectangular-shape that defines the location and size of an object in an image. The two rest zones 101 and 102 of Wei, as disclosed in col. 3, line 64- col. 4, line 7, “may be disposed at two sides of the running track 12, which provide flat and stable surfaces for enabling the user A to stand on the rest zones 101 and 102 to take a rest… the light emitter 142 may be fixed on the solid case 104 and emit light to cover the two rest zones 101, 102 and the running zone 103, and the light sensor 140 detects a position and a speed of the user A according to a reflected light from the user A,” and therefore teach, in the broadest reasonable interpretation of the claim, the background area of the claimed invention. In light of the functional language, Nguyen teaches the limitation of the claimed invention, “processor is configured to perform motion detection on a background area in the sensing image to detect the moving object in the background area,” as Nguyen discloses in paragraphs 0021-0022, “A runner 105 ss exercising (e.g., running or walking) on a treadmill 100. The treadmill 100 monitors a rear proximity using object detection sensors 120 located on a deck 110 of the treadmill 100. For example, the sensors 120 are configured or positioned to monitor a rear area 130 having a certain area or width that extends behind the deck 110 of the treadmill 100. When a ball 135 or other object (e.g., a pet, a toy, and so on), enters the rear area 130 monitored by the object detection sensors 120, the sensors 120 can detect the presence of the ball 135 and/or whether the ball is moving towards the deck 110 of the treadmill 100. In response to the detection, the treadmill 100, via an associated control system or safety system, can cause or trigger an alarm, warning, or notification that is presented to the runner 105 and/or shut down or slow the treadmill (via an associated safety controller or safety key mechanism)”.
Regarding claim 10, Wei shows a speed control method of a treadmill (Wei, “The present invention relates to a treadmill, which is adapted to control the speed of a running track based on an operator's (runner) speed or position. The present disclosure uses a light sensor to detect a position of a runner on the treadmill and correspondingly controls the operating status of the treadmill, i.e., speed of a running track on the treadmill”, col. 3, lines 44-50), comprising: generating a sensing image through an event-based vision sensor disposed on the treadmill, wherein each pixel in the sensing image generated by the event-based vision sensor represents variation of light intensity (Wei, “the light sensor 140 may be an image sensor, such as CMOS or CCD, or may include a plurality of photodiodes. Specifically, the light sensor 140 is configured to obtain a captured image including at least one object image indicating position of the user A… When an incident light travels to a part of the body of the user A, such as legs or shoes, the incident light will be scattered, diffused, and reflected, and the light sensor 140 may detect image by receiving the scattered light, diffused light, and reflected light and obtain the captured image… Specifically, the object image could be a dot or a group of strips depends on the type of light emitted from the light emitter 142. In addition, the feature of the at least one object image includes at least one of a size and a variation frequency of a pattern”, col. 4, lines 10-36; It is well known in the art that CMOS and CCD sensors are designed to detect and measure changes in light intensity. More specifically, photodiodes are designed to detect and measure light intensity); performing runner detection on the sensing image (Wei, “Specifically, the light sensor may include one or more processors or microcontrollers for processing and identifying the object image in the captured image”, col. 4, lines 47-50); in response to determining that a runner is detected from the sensing image, inputting the sensing image to a depth estimation model, obtaining a depth map output by the depth estimation model and obtaining position information of the runner relative to a running belt of the treadmill according to the depth map; controlling a running speed of the running belt according to the position information of the runner (Wei, “Specifically, the light sensor may include one or more processors or microcontrollers for processing and identifying the object image in the captured image. When the light sensor 140 identifies the at least one object image in the captured image, the light sensor 140 generates information related to the object images for the control unit 106 to adjust the speed of the running track 12… the control unit 106 may also include an image-processing unit for receiving the captured image and calculating the characteristic properties of the object image according to the captured image. The control unit 106 may further be configured to adjust the speed of the running track 12 according to the characteristic properties of the object image accordingly… For example, when the light sensor 140 detects that the user A is located at the near zone NZ, the control unit 106 may be configured to accelerate the running track 12; when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”, col. 4, lines 47-col. 5, line 28; The depth estimation model is shown by the detection of the user within a specific zone of Wei with is relative to the running track of Wei as the specification of the claimed invention describes the depth estimation model, in paragraph 0027, “the processor 130 inputs the sensing image to the depth estimation model to obtain depth information of the runner, thereby obtaining the position information of the runner on the running belt 112”. See FIG. 1 below).
Wei fails to show, the event-based vision sensor comprises a dynamic vision sensor; the depth estimation model comprising a monocular depth estimation model trained using a deep learning model; and specifically in response to detection of a moving object, inputting the sensing image including the moving object to the depth estimation model comprising the monocular depth estimation model, obtaining the depth map outputted by the estimation model, obtaining position information of the moving object, determining a second distance between the moving object and a reference position according to the depth map, and controlling the running speed of the running belt according to the position information of the moving object.
However, Lim, from the same field of endeavor, discloses a method for checking out sports motion using event-based vision sensor and apparatus for the same and teaches a dynamic vision sensor (Lim, “The event-based vision sensor used in an embodiment of the present invention is an apparatus for generating at least one event signal by detecting a part of an object in which a motion is generated. For example, a dynamic vision sensor is a commercially available event-based sensor. Instead of transmitting a frame sequence, which is a continuous signal of a still image, a dynamic vision sensor detects a change in light intensity to identify a specific scene in the scene when an event occurs. Only a change in the luminance (e.g., event) of the pixel at the position is transmitted. In other words, the output of the dynamic vision sensor is a stream of events where each event is associated with a particular state”, page 3).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have replaced the detection module which includes a light emitter and a light sensor of Wei with the dynamic vision sensor of Lim as Lim discloses in page 3, “the event signal output from the dynamic vision sensor carries only the information of the pixel detected by the user's movement, it can analyze the image much faster and more accurately than the conventional camera transmitting thousands of frames per second”. In light of this modification, the event signal output of Lim teaches the sensing image of the claimed invention. Lim also discloses on page 3, “Conventional vision sensors collect the scene as a sequence of frames from the moving image and include all photographic elements including pixels within one frame. In the frame sequence, consecutive frames before and after are collected in units of time, and there is overlapped pixel information without change. As such, storing and processing redundant pixel information frame by frame wastes storage space, processing time, and battery power” which disclose the function of the preferred light sensors, CMOS and CCD sensors, of Wei, making this modification obvious. Moreover, dynamic vision sensors generate a sensing image, wherein each pixel in the sensing image generated by the event-based vision sensor represents variation of light intensity, thereby teaching the limitation of the claimed invention “generating a sensing image, wherein each pixel in the sensing image generated by the event-based vision sensor represents variation of light intensity” with this modification.
Furthermore, Brown, a teaching reference showing a similar problem of estimating the position of a user with a capture device, discloses a method and system for monocular depth estimation of persons and teaches a monocular depth estimation model (Brown, “the human joint 3D estimation system 110 may comprise a capture device 120 to capture a digital image, a 2D skeleton detector 130, which may detect and crop humans in an image and localize human joint positions in the image, a 2D joint heatmap generator 140, which may generate a positional heatmap for each kind of joint to be localized, a depth heatmap estimator 150, which may generate 1 D depth heatmaps for each kind of localized joint, and a 3d joint constructor 160, which may select a depth value for each kind of joint from the depth heatmaps and combine this information with the 2D joint positions to produce 3D joint positions”, paragraph 0017; The human joint 3D estimation system of Brown teaches the monocular depth estimation model of the claimed invention) trained using a deep learning model (Brown, “A depth heatmap estimator may be embodied by a trained machine learning model. This model may be a convolutional neural network (CNN), or alternatively a RNN, random forest, or deep neural network”, paragraph 0026).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have replaced the image-processing unit of Wei that detects the user within a specific zone of Wei with the human joint estimation system of Brown as Brown discloses in paragraphs 0003-0005, “Depth sensing hardware is often expensive and limiting in its use, such as having a limited range and may only be used inside… existing approaches often exhibit limited accuracy and robustness, especially in challenging poses, environments and conditions. Thus, there exists the need for a better approach to monocular RGB estimation of human joint depths”. Brown also discloses in paragraph 0018, “A capture device 120 may represent any device or method for acquiring a digital image or video frame”, and dynamic vision sensors are devices for acquiring digital visual data, making this modification obvious. Moreover, Brown discloses in paragraph 0037-0040, “The 3D joint constructor 160 may accept 2D joint positions for one detected person, for example from the 2D skeleton detector 130, and corresponding joint depth heatmaps, produced by the depth heatmap estimator 150, and output 3D joint positions… By combining the 2D joint heatmap with the depth estimation for each kind of joint, a 3D location for each joint can be estimated. A rendering 740 of a skeleton based on these 3D locations is shown”. Therefore, Wei, in view of Brown teaches the limitation of the claimed invention “the processor obtaining a depth map output by the depth estimation model and obtaining position information of the runner relative to the running belt” with the 3D locations for each joint of Brown.
However, Nguyen, from the same field of endeavor, teaches in response to detection of a moving object, determine a second distance between the moving object and a reference position (Nguyen, “the module 202 can receive distance information from an object, and determine the object is within a certain area or proximity of the exercise machine. Further, using multiple detected locations, the module 202 can determine if the object is moving towards the machine and/or acceleration/decelerating with respect to the machine”, paragraph 0039; The distance information from the moving object of Nguyen teaches the second distance between the moving object and a reference position) so as to control the running speed of the running belt according to the position information of the moving object (Nguyen, “In some embodiments, the operation modification module 204 is configured and/or programmed to modify a current operation of the exercise machine based on detecting the object is proximate to the exercise machine… the module 204 can perform an action to control a mechanical operation of the exercise machine, such as by transmitting instructions to a controller 220 or control system of the exercise machine. For example, when the exercise machine is a treadmill, the operation modification module 204 causes the controller 220 to slow or stop the movement of the running surface of the treadmill”, paragraphs 0041-0042).
Nguyen also discloses in paragraph 0040, “Of course, the module 202 can utilize various combinations of sensors and captured information, as described in greater detail herein. For example, the object detection module 202 can (1) receive information from a time-of-flight sensor that detects the object is within the area surrounding the exercise machine and (2) a computer vision (CV) sensor that determines the object detected to be within the area surrounding the exercise machine is moving towards the exercise machine and/or a millimeter wave (mmWave) sensor that determines the object detected to be within the area surrounding the exercise machine is moving towards the exercise machine”. The time-of-flight sensors of Nguyen are comparable to the event-based vision sensor of Lim such that both sensors detect motion. The OpenCV techniques of Nguyen disclosed in paragraphs 0031 and 0032 of Nguyen are used for depth estimation techniques, as well as, millimeter wave sensors of Nguyen disclosed in paragraphs 0033-0035. Therefore, the module of Nguyen is comparable to the human joint 3D estimation system of Brown such that both are models for depth estimation based on data collected by sensors that detect motion. More specifically, as seen in paragraph 0033 of Nguyen, “The mmWave sensors utilize a Frequency Modulated Continuous Wave (FMCW) mechanism that transmits and receives chirps (e.g., waves at 60 Ghz or 24 Ghz) and can capture three dimensions of information (e.g., velocity, range or distance, and angle) for an object proximate to the exercise machine”, the three dimensions of information, including the distance for an object proximate to the machine, is comparable to the depth map of the claimed invention.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have the modified treadmill of Wei also control the speed of a running track upon the detection of a moving object according to the methods already disclosed by Wei, the dynamic vision sensor of Lim, and the human joint 3D estimation system of Brown to further “provide a durable and safe treadmill” (Nguyen, col. 1, lines 53-54). Wei already discloses controlling the speed of a running track of a treadmill based on the speed or position of the runner/user. Nguyen simply improves on this by controlling the speed of the running surface of the treadmill if a moving object is heading towards the treadmill. Both Brown and Nguyen disclose utilizing sensors for detecting motion to determine distances through comparable processes, making this modification obvious. More specifically, it would have been obvious for the processor, as shown by Wei, to be configured to input the sensing image including the moving object, taught by Nguyen, to the human joint 3D estimation system, obtain the depth map output by the human joint 3D estimation system, obtain position information of the moving object, as Wei already shows this process for detecting the position of a runner on the belt. This modification simply utilizes the combination sensors of Nguyen to process, in the same manner as that of the runner, for detecting the position of a moving object near the treadmill.
Regarding claim 11, Wei, in view of Lim, Brown, and Nguyen, teaches the speed control method according to claim 10, wherein the step of performing the runner detection on the sensing image comprising: performing person detection on the sensing image and obtaining a person bounding box on the sensing image; and determining whether the runner is detected according to whether the person bounding box is located within a predetermined area on the sensing image (Wei, “As shown in FIG. 3, the running zone 103 may be divided into several zones, and when the light sensor 140 identifies which zone the user A is located, the control unit 106 may control the speed of the running track 12 directly. For example, the running track 12 may be divided into three zones, near zone NZ, middle zone MZ, and far zone FZ with respect to positions on the running zone 103 along a running direction D1 of the user. It should be noted that the first row R1 and the neighboring rows correspond to a region of the running track 12 that is near the front of the treadmill M in the example of the 1D image and 2D image, such as the near zone NZ in FIG. 3. In other embodiment, the three zones could only correspond to particular rows in the captured image, such as rows R1-R3 corresponding to near zone NZ, rows R4-R6 corresponding to middle zone MZ, and rows R7-R9 corresponding to far zone FZ”, col. 4, lines 65-67, col. 5, lines 1-16; In the broadest reasonable interpretation of the claim, the running zone 103 disclosed by Wei, as seen in FIGS. 2A and 3 below, show at least one bounding box of the claimed invention as it is well known in the art that in image processing, a bounding box is a rectangular-shape that defines the location and size of an object in an image).
Regarding claim 12, Wei, in view of Lim, Brown, and Nguyen, teaches the speed control method according to claim 11, wherein the step of determining whether the runner is detected according to whether the person bounding box is located within the predetermined area on the sensing image comprising: determining that the runner is not detected if the person bounding box is not located within the predetermined area on the sensing image (Wei, “On the other hand, if the light sensor 106 detects that the user is located at one or both of the two rest zones 101 and 102, the control unit 106 may be configured to stop the running track 12”, col. 5, lines 28-31; When the user is in not in the running zone, i.e. either of the rest zones, and the treadmill stops the track, Wei therefore does not detect a user in the predetermined area of the running zone); and determining that the runner is detected if the person bounding box is located within the predetermined area on the sensing image (Wei, “The present embodiment further provides several cases for the control unit 106 to adjust the speed of the running track 12 according to the position of the user A. For example, when the light sensor 140 detects that the user A is located at the near zone NZ, the control unit 106 may be configured to accelerate the running track 12; when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”, col. 5, lines 17-27; Wei shows the location of a user in the running zone and the adjustment of the speed shows detection of the user).
Regarding claim 13, Wei, in view of Lim, Brown, and Nguyen, teaches the speed control method according to claim 10, wherein the step of in response to determining that the runner is detected from the sensing image, inputting the sensing image to the depth estimation model and obtaining the position information of the runner relative to the running belt comprising: in response to determining that the runner is detected from the sensing image, inputting the sensing image to the depth estimation model and obtaining the depth map output by the depth estimation model (Wei, “In more detail, the light sensor 140 obtains a captured image including at least one object image indicating position of the user A, and the captured image could be formed as a 2D image or an 1D image, as shown in FIGS. 2A and 2B”, col. 4, lines 37-41; The at least one object image of Wei shows the runner being detected from the sensing image of the claimed invention. The modified treadmill of Wei teaches the processor inputting the sensing image to the dynamic vision sensors and obtaining the 3D locations for each joint with the human joint 3D estimation system of Brown); and determining a first distance between the runner and the reference position according to the depth map (Wei, “As shown in FIG. 3, the running zone 103 may be divided into several zones, and when the light sensor 140 identifies which zone the user A is located, the control unit 106 may control the speed of the running track 12 directly. For example, the running track 12 may be divided into three zones, near zone NZ, middle zone MZ, and far zone FZ with respect to positions on the running zone 103 along a running direction D1 of the user. It should be noted that the first row R1 and the neighboring rows correspond to a region of the running track 12 that is near the front of the treadmill M in the example of the 1D image and 2D image, such as the near zone NZ in FIG. 3. In other embodiment, the three zones could only correspond to particular rows in the captured image, such as rows R1-R3 corresponding to near zone NZ, rows R4-R6 corresponding to middle zone MZ, and rows R7-R9 corresponding to far zone FZ”, col. 4, line 67-col. 5, lines 1-16; The first distance between the runner and the reference position is shown by Wei with the user in a respective zone of Wei).
Regarding claim 14, Wei, in view of Lim, Brown, and Nguyen, teaches the speed control method according to claim 13, wherein the step of controlling the running speed of the running belt according to the position information of the runner comprising: controlling the running speed of the running belt to decrease if the first distance is greater than a first threshold value (Wei, “When the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”, col. 5, lines 22-28; The threshold value of the claimed invention is shown by the middle zone of Wei); and maintaining the running speed of the running belt if the first distance is not greater than the first threshold value (Wei, “when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12”, col. 5, lines 22-25).
Regarding claim 18, Wei, in view of Lim, Brown, and Nguyen, teaches the speed control method according to claim 10.
Wei fails to explicitly teach, wherein the step of controlling the running speed of the running belt according to the position information of the moving object comprising: controlling the running speed of the running belt to decrease if the second distance is less than a second threshold value; and maintaining the running speed of the running belt if the second distance is not less than the second threshold value.
However, Nguyen teaches, wherein the step of controlling the running speed of the running belt according to the position information of the moving object comprising: controlling the running speed of the running belt to decrease if the second distance is less than a second threshold value (Nguyen, “In some cases, a monitored region can include multiple sub-regions, and the treadmill 100 can perform different actions that are based on the detection of objects within one or more of the sub-regions… a closer sub-region 140 can be associated with a slowing of the treadmill 100”, paragraph 0023); and maintaining the running speed of the running belt if the second distance is not less than the second threshold value (Nguyen, “if a region has three sub-regions (as depicted in FIG. 1B), the sub-regions can each be associated with a different action. For example, a sub-region 150 farthest from the treadmill 100 can be associated with a warning presented to the runner 105”, paragraph 0023; The sub-region of Nguyen which only outputs a warning teaches the maintaining of the running speed of the running belt as the closer sub-region and the inner sub-region, also disclosed in paragraph 0023, require the slowing or stopping of the treadmill. See FIG. 1B below).
PNG
media_image5.png
294
181
media_image5.png
Greyscale
Nguyen
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified treadmill of Wei to specifically adjust the speed of the treadmill dependent on the position of the moving object, taught by Nguyen, as Wei already discloses variable speed adjustment based on position of the user. Specifically, in col. 5, lines 19-28, Wei discloses “For example, when the light sensor 140 detects that the user A is located at the near zone NZ, the control unit 106 may be configured to accelerate the running track 12; when the light sensor 140 detects that the user A is located at the middle zone MZ, the control unit 106 may be configured to maintain the speed of the running track 12. When the light sensor 140 detects that the user A is located at the far zone FZ, the control unit 106 may be configured to deaccelerate the running track 12”. Therefore, it would be obvious to apply the same variable speed adjustment for the treadmill based on the position of the moving object as well.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 10 have been considered but are moot because the new ground of rejection relies on Lim (KR102055146B1) and Brown (WO2020250046A1) to teach the newly added limitations. Furthermore, applicant's arguments filed 12/08/2025 regarding newly added claim 19 have been fully considered but they are not persuasive as Wei (US10,953,281B2), in view of Nguyen (US2022/0355179A1), teaches the limitations of claim 19. See 103 rejections above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to J LOBERIZA whose telephone number is (571)272-4741. The examiner can normally be reached 8am - 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, LoAn Jimenez can be reached at 571-272-4966. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACQUELINE N L LOBERIZA/Examiner, Art Unit 3784
/LOAN B JIMENEZ/Supervisory Patent Examiner, Art Unit 3784
Prosecution Insights