DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Claim objection is withdrawn based on Applicant’s amendments. New objection to claim 19 is provided below which was raised because of amendments to the claims. Applicant's arguments filed on 04/21/2026 with respect to claims 1, 4, 8, 11, 13-16, 18, 19, 22, and 24-28 have been fully considered but they are not persuasive or moot in view of new ground of rejection provided below which was necessitated based on Applicant’s amendments to the claims.
Claim Objections
Claim 19 is objected to because of the following informalities: “receiving, over time, reflections a different second light patterns from the plurality of objects” should read “receiving, over time, reflections of a different second light patterns from the plurality of objects”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 16, 18, 19, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed).
Regarding Claim 1, Tokunaga teaches a robot comprising:
a light-emitting device comprising a plurality of separately-controllable light emitting regions,
each of the plurality of separately-controllable light emitting regions comprising a plurality of separately-controllable light-emitting elements (See at least Para [0046] “… The pattern is constituted by a combination of a plurality of irradiation lights (a plurality of configuration elements) irradiated by the irradiator.” Para [0081] “The irradiator 40 can be configured by a projector that emits a pattern. The pattern irradiated by the irradiator 40 is configured by a combination of a plurality of configuration elements (a plurality of irradiation lights)…”);
a sensor (See at least Para [0046] “… Also in the structured light method, the depth information of the object surface can be estimated by triangulation on the basis of a detection result of the pattern by the sensor (camera) and a positional relationship between an irradiator and the sensor. The pattern is constituted by a combination of a plurality of irradiation lights (a plurality of configuration elements) irradiated by the irradiator.”, Para [0047] “Furthermore, there is known a technique (so-called Active Stereo method) of estimating depth information of an object surface irradiated with irradiation light by triangulation on the basis of a positional relationship between an irradiator and a sensor (The sensor is typically a stereo camera.), an irradiation position of the irradiation light by the irradiator (with reference to the irradiator), and a detection position of the irradiation light by the sensor (with reference to the sensor)…”, Para [0095] “Alternatively, the self-position estimation unit 50 may estimate the self-position and orientation in the real space using a monocular camera. For example, an a case where a monocular camera as used, a change amount relative to the initial state of a self-position and orientation can be estimated on the basis of which position a predetermined point in the real space is detected at each of a plurality of positions by the monocular camera and a movement amount between the plurality of positions of the monocular camera…”); and
a controller configured to control the light-emitting device and the sensor to perform a space-specific object recognition process in at least one three dimensional (3D) space of a plurality of 3D spaces outside of the robot, each 3D space of the plurality of 3D spaces corresponding to a different one of the plurality of separately-controllable light emitting regions of the light-emitting device (See at least Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40. More specifically, the irradiation control unit 131 outputs an irradiation instruction signal for instructing irradiation of a pattern to the irradiator 40. In the first embodiment of the present disclosure, it is mainly assumed that the irradiation control unit 131 controls the irradiator 40 so that the pattern is constantly irradiated by the irradiator 40. However, as described in the following embodiment, the irradiation control unit 131 may control the irradiator 40 so that the switching is performed between irradiation and stop of the pattern.”, Para [0113] “More specifically, a positional relationship between the IR camera 30 and the irradiator 40 is grasped in advance by the control unit 130…”, Para [0126] “More specifically, a positional relationship between the DVS camera 20 and the irradiator 40 is grasped in advance by the control unit 130.”, Para [0164] “…The control unit 130 includes the irradiation control unit 135, a depth information estimating unit 136, an integrated processing unit 133, and a recording control unit 134…”, Para [0135] “The integrated processing unit 133 estimates three-dimensional information P(n), P(k), and P(m) on the world coordinates by inverse projection processing on the basis of the depths D(n), D(k), and D(m), and the self-position and orientation information Pcamera (n), Pcamera (k), and Pcamera(m). For example, such three-dimensional information corresponds to the three-dimensional position (three-dimensional position of each configuration element) of the surface of the object 70 irradiated with the irradiation light at each of the times t(n), t(k), and t(m) (S16), and can be expressed as a three-dimensional point group on the same world coordinate.”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”), the space-specific object recognition process comprising:
controlling, over time, different light-emitting elements of one of the plurality of separately-controllable light-emitting elements that corresponds to the 3D space to emit two or more light patterns into a corresponding one 3D space (See at least Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40…”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”, Para [0111] “The IR camera 30 captures a first frame (captured image) and outputs the captured first frame to the information processing apparatus 10. The depth information estimating unit 132 detects a part or all of the pattern (a plurality of configuration elements) on the basis of the first frame output from the IR camera 30.”, Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”, Para [0074] “… For example, the information processing system 1 may be moved by a robot and the like, or may be provided in a moving body (for example, a vehicle, a person, and the like)…”),
wherein the two or more light patterns comprise one of two or more predefined light patterns or two or more random light patterns (See at least Para [0082] “In the embodiment of the present disclosure, a case here a pattern (random dot pattern) in which a plurality of dots (points or circles) is scattered is used as the pattern irradiated by the irradiator 40 is mainly assumed” Para [0083] “… Alternatively, a random color pattern with visible light may be used as the pattern irradiated by the irradiator 40.”),
receiving, over time, reflections of the two or more light patterns from the object (See
at least Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”,),
performing a comparison of the reflections … so as to generate a first comparison result (See at least Para [0007] “… the first detection position being output from a first sensor that detects a position where a luminance change greater than or equal to a first threshold value has occurred…”), and
based on the first comparison result indicating … recognition that is higher than or equal to a preset value, recognizing that the object is present in the 3D space (See at least Para [0153] “As described above, in the first embodiment of the present disclosure, the depth information estimating unit 132 estimates the first depth information on the basis of the first detection position of the irradiation light by the irradiator 40 output from the DVS camera 20 that detects the position where the luminance change greater than or equal to the first threshold value has occurred. Then, the integrated processing unit 133 estimates the three-dimensional information on the basis of the first depth information and the position and orientation information of the first sensor at each of the plurality of times.”).
However, Tokunaga does not explicitly spell out … to object class specific information … an object class specific probability of recognition …
Shadeed teaches … to object class specific information (See at least Page 505 Col 1 Para
1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”)… an object class specific probability of recognition (See at least Page 505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Shadeed with Tokunaga and include the feature of object class specific probability of recognition, thereby provide precise calculation for improved safety (See at least Page 503 Col 1 “1. INTRODUCTION … 3) Making vehicles safer through improved active and passive safety measures and technology.”).
Regarding Claim 16, modified Tokunaga teaches all the elements of claim 1. Tokunaga further teaches the robot of claim 1, wherein, when the object recognized as being present in the 3D space is one of a plurality of objects recognized as being present in the 3D space, the space-specific object recognition process further comprises: …
based on the determination result, controlling, over time, the different light-emitting
elements of one of the plurality of separately-controllable light-emitting elements that corresponds to the 3D space to emit the two or more light patterns into the one 3D space (See at least Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40…”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”, Para [0111] “The IR camera 30 captures a first frame (captured image) and outputs the captured first frame to the information processing apparatus 10. The depth information estimating unit 132 detects a part or all of the pattern (a plurality of configuration elements) on the basis of the first frame output from the IR camera 30.”, Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”, Para [0074] “… For example, the information processing system 1 may be moved by a robot and the like, or may be provided in a moving body (for example, a vehicle, a person, and the like)…”) in accordance with:
However, Tokunaga does not explicitly spell out …
determining whether or not types of the plurality of objects are the same so as to generate a determination result, and …
a first object recognition mode when the plurality of objects existing in the 3D space are of different types , or
a second object recognition mode when the plurality of objects existing in the 3D space are of the same type.
Shadeed teaches …
determining whether or not types of the plurality of objects are the same so as to generate a determination result (See at least Page 505 Col 1 Para 1 “Our system does the same via detecting the light distributions of the oncoming/leading vehicles and classifying it in real-time using neural network and fuzzy logic supported with a prototypical database. For the reason that we are dealing with an opening environment like vehicle's space; neither one technique nor one set of hypotheses is applicable to be used to detect different types of objects. Therefore we developed our system based on separating objects in three categories Taillamps, Headlamps and Lane markings; which are probably appear mostly in the traffic situations and may be considered as the relevant targets to our system. The algorithm is designed for parallel processing in order to be suitable to be implemented later on FPGA. Figure 5 shows the main structure of the algorithms. The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification.”), and …
a first object recognition mode when the plurality of objects existing in the 3D space are of different types (See at least Page 506 Col 2 Para 2 “2) LED-Array Headlamp - … the LED-array
headlamp does not need moving elements to generate different light distributions. Instead, the light sources are addressed directly. The light distributions are generated by creating an image of a matrix of LED-chips. The possibility of individually controlling each LED-chip of the matrix allows the generation of different shapes of light. Activating or deactivating single LED-chips of the matrix can easily realize assisting light functions; for example a glare-free high-beam function could be generated by switching off one or several of the LED-chips that illuminate the area of the oncoming traffic. Using a PWM (Pulse Width Modulation) principal to drive the LED-chips makes it possible to produce different levels of brightness, which enable us to adjust the light intensity according to the road illumination. Activating single chips that contribute to the light distribution above the cut-off line could be used to realize a marking light function. Figure 10 depicts a LED-array prototype as well as a low-beam and high-beam light distribution generated with this array. The possibility of separately controlling the brightness of each LED-chip of the matrix allows the generation of driver-specific light distributions.”, discloses adjusting the light intensity according to the road illumination which is construed as controlling the light source unit to recognize the objects existing in the space in a first object recognition mode or a second object recognition mode based on a result of the determination), or
a second object recognition mode when the plurality of objects existing in the 3D space are of the same type (See at least Page 506 Col 2 Para 2 “2) LED-Array Headlamp - … the LED-
array headlamp does not need moving elements to generate different light distributions. Instead, the light sources are addressed directly. The light distributions are generated by creating an image of a matrix of LED-chips. The possibility of individually controlling each LED-chip of the matrix allows the generation of different shapes of light. Activating or deactivating single LED-chips of the matrix can easily realize assisting light functions; for example a glare-free high-beam function could be generated by switching off one or several of the LED-chips that illuminate the area of the oncoming traffic. Using a PWM (Pulse Width Modulation) principal to drive the LED-chips makes it possible to produce different levels of brightness, which enable us to adjust the light intensity according to the road illumination. Activating single chips that contribute to the light distribution above the cut-off line could be used to realize a marking light function. Figure 10 depicts a LED-array prototype as well as a low-beam and high-beam light distribution generated with this array. The possibility of separately controlling the brightness of each LED-chip of the matrix allows the generation of driver-specific light distributions.”, discloses adjusting the light intensity according to the road illumination which is construed as controlling the light source unit to recognize the objects existing in the space in a first object recognition mode or a second object recognition mode based on a result of the determination).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Shadeed with Tokunaga and include the feature of determining whether or not types of the plurality of objects are the same so as to generate a determination result, and a first object recognition mode when the plurality of objects existing in the 3D space are of different types or a second object recognition mode when the plurality of objects existing in the 3D space are of the same type, thereby provide precise calculation for improved safety (See at least Page 503 Col 1 “1. INTRODUCTION … 3) Making vehicles safer through improved active and passive safety measures and technology.”).
Regarding claim 18, modified Tokunaga teaches all the elements of claim 16.
However, Tokunaga does not explicitly spell out the robot of claim 16, wherein the first object
recognition mode is a mode of extracting different object class probabilities for each type of the different types.
Shadeed teaches the robot of claim 16, wherein the first object recognition mode is a mode of
extracting different object class probabilities for each type of the different types (See at least Page 505 Col 1 Para 1 “Our system does the same via detecting the light distributions of the oncoming/leading vehicles and classifying it in real-time using neural network and fuzzy logic supported with a prototypical database. For the reason that we are dealing with an opening environment like vehicle's space; neither one technique nor one set of hypotheses is applicable to be used to detect different types of objects. Therefore we developed our system based on separating objects in three categories Taillamps, Headlamps and Lane markings; which are probably appear mostly in the traffic situations and may be considered as the relevant targets to our system. The algorithm is designed for parallel processing in order to be suitable to be implemented later on FPGA. Figure 5 shows the main structure of the algorithms. The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Shadeed and include the feature of the first object recognition mode being a mode of extracting different object class probabilities for each type of the different types, thereby provide precise calculation for improved safety (See at least Page 503 Col 1 “1. INTRODUCTION … 3) Making vehicles safer through improved active and passive safety measures and technology.”).
Regarding Claim 19, modified Tokunaga teaches all the elements of claim 18. Tokunaga further teaches …
controlling, over time, different light-emitting elements of the plurality of separately-
controllable light-emitting elements that corresponds to the 3D space to emit different light patterns into the one 3D space (See at least Para [0068] “Furthermore, in order to estimate the three-dimensional shape of the object surface, it is necessary to add depth information at each of a plurality of times…”, Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40…”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”, Para [0111] “The IR camera 30 captures a first frame (captured image) and outputs the captured first frame to the information processing apparatus 10. The depth information estimating unit 132 detects a part or all of the pattern (a plurality of configuration elements) on the basis of the first frame output from the IR camera 30.”, Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”, Para [0074] “… For example, the information processing system 1 may be moved by a robot and the like, or may be provided in a moving body (for example, a vehicle, a person, and the like)…”), and
receiving, over time, reflections a different second light patterns from the plurality of objects (See at least Para [0068] “Furthermore, in order to estimate the three-dimensional shape of the object surface, it is necessary to add depth information at each of a plurality of times…”, Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40…”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”, Para [0111] “The IR camera 30 captures a first frame (captured image) and outputs the captured first frame to the information processing apparatus 10. The depth information estimating unit 132 detects a part or all of the pattern (a plurality of configuration elements) on the basis of the first frame output from the IR camera 30.”, Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”, Para [0151] “Note that adjacent surfaces of objects existing in reality are continuous. Therefore, as illustrated in FIG. 8, the complementary processing performed by the integrated processing unit 133 may include processing of estimating the position of the shape corresponding to the disappearance point 81 such that the position of the three-dimensional shape corresponding to the detection point 82 and the position of the three-dimensional shape corresponding to the disappearance point 81 are continuous.”),
performing a second comparison of the reflections of the different second light patterns to, … that is higher than or equal to a type-specific threshold, recognizing that the object … is present in the 3D space (See at least Para [0153] “As described above, in the first embodiment of the present disclosure, the depth information estimating unit 132 estimates the first depth information on the basis of the first detection position of the irradiation light by the irradiator 40 output from the DVS camera 20 that detects the position where the luminance change greater than or equal to the first threshold value has occurred. Then, the integrated processing unit 133 estimates the three-dimensional information on the basis of the first depth information and the position and orientation information of the first sensor at each of the plurality of times.”).
However, Tokunaga does not explicitly spell out the robot of claim 18, wherein, in the first object recognition mode, the space-specific object recognition process further comprises:
extracting the object class probabilities for each type of the different types ; and …
object class specific information so as to generate a different object class comparison results … and based on an average of the different object class comparison results indicating an object class specific probability of recognition … recognizing that the object class …
Shadeed teaches the robot of claim 18, wherein, in the first object recognition mode, the space-specific object recognition process further comprises:
extracting the object class probabilities for each type of the different types (See at least Page
505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”); and … object class specific information so as to generate a different object class comparison results (See at least Page 505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”) … and based on an average of the different object class comparison results indicating an object class specific probability of recognition (See at least Page 505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”) … recognizing that the object class (See at least Page 505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”) …
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Shadeed with Tokunaga and include the feature of object class specific probability of recognition, thereby provide precise calculation for improved safety (See at least Page 503 Col 1 “1. INTRODUCTION … 3) Making vehicles safer through improved active and passive safety measures and technology.”).
Regarding Claim 22, modified Tokunaga teaches all the elements of claim 16. Tokunaga further teaches …
controlling, over time, different light-emitting elements of the plurality of separately-
controllable light-emitting elements that corresponds to the 3D space to emit different light patterns into the one 3D space (See at least Para [0068] “Furthermore, in order to estimate the three-dimensional shape of the object surface, it is necessary to add depth information at each of a plurality of times…”, Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40…”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”, Para [0111] “The IR camera 30 captures a first frame (captured image) and outputs the captured first frame to the information processing apparatus 10. The depth information estimating unit 132 detects a part or all of the pattern (a plurality of configuration elements) on the basis of the first frame output from the IR camera 30.”, Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”, Para [0074] “… For example, the information processing system 1 may be moved by a robot and the like, or may be provided in a moving body (for example, a vehicle, a person, and the like)…”), and
receiving, over time, reflections a different second light patterns from the plurality of objects (See at least Para [0068] “Furthermore, in order to estimate the three-dimensional shape of the object surface, it is necessary to add depth information at each of a plurality of times…”, Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40…”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”, Para [0111] “The IR camera 30 captures a first frame (captured image) and outputs the captured first frame to the information processing apparatus 10. The depth information estimating unit 132 detects a part or all of the pattern (a plurality of configuration elements) on the basis of the first frame output from the IR camera 30.”, Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”, Para [0151] “Note that adjacent surfaces of objects existing in reality are continuous. Therefore, as illustrated in FIG. 8, the complementary processing performed by the integrated processing unit 133 may include processing of estimating the position of the shape corresponding to the disappearance point 81 such that the position of the three-dimensional shape corresponding to the detection point 82 and the position of the three-dimensional shape corresponding to the disappearance point 81 are continuous.”),
performing a second comparison of the reflections of the different second light patterns to, … that is higher than or equal to a type-specific threshold, recognizing that the object … is present in the 3D space (See at least Para [0153] “As described above, in the first embodiment of the present disclosure, the depth information estimating unit 132 estimates the first depth information on the basis of the first detection position of the irradiation light by the irradiator 40 output from the DVS camera 20 that detects the position where the luminance change greater than or equal to the first threshold value has occurred. Then, the integrated processing unit 133 estimates the three-dimensional information on the basis of the first depth information and the position and orientation information of the first sensor at each of the plurality of times.”).
However, Tokunaga does not explicitly spell out the robot of claim 16, in the second object recognition mode, the space-specific object recognition process further comprises:
extracting the object class probabilities for each type of the different types ; and …
object class specific information so as to generate a different object class comparison results … and
based on an average of the different object class comparison results indicating an object class specific probability of recognition … recognizing that the object class …
Shadeed teaches the robot of claim 18, wherein, in the first object recognition mode, the space-specific object recognition process further comprises:
extracting a class probability for the plurality of objects (See at least Page
505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”); and … object class specific information so as to generate a different object class comparison results (See at least Page 505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”) … and
based on an average of the different object class comparison results indicating an object class specific probability of recognition (See at least Page 505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”) … recognizing that the object class (See at least Page 505 Col 1 Para 1 “The system capture a colour frame from the image sensor and save it in three different formats, colour, grey, and binary image. Each frame-type is associated to a fast one pass customised filter, which is designed mainly to detect one type of objects at a time. After successful detection, the data are feed to three classifier-agents to determine the class of the object. Each agent has a weight. Each weight is multiplied with the estimated classification probability evaluated from the agent, and then the result is summed and normalized to estimate the percentage of trustiness of classification…”) …
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Shadeed with Tokunaga and include the feature of object class specific probability of recognition, thereby provide precise calculation for improved safety (See at least Page 503 Col 1 “1. INTRODUCTION … 3) Making vehicles safer through improved active and passive safety measures and technology.”).
Claim(s) 4 is rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed), and further in view of Kroyan et al. (US 20170266574 A1) (Hereinafter Kroyan).
Regarding Claim 4, modified Tokunaga teaches all the elements of claim 1.
However, Tokunaga does not explicitly spell out the robot of claim 1, further comprising:
a memory that stores information that includes information related to the plurality of 3D spaces
and information related to a corresponding plurality of 3D space-specific light patterns,
wherein, when information sensed by the sensor corresponds to the information in the memory, the emitted two or more light patterns is the two or more predefined light patterns, and
wherein, when information sensed by the sensor does not correspond to the information in the memory, the two or more light patterns is the two or more random light patterns.
Kroyan teaches the robot of claim 1, further comprising:
a memory that stores information that includes information related to the plurality of 3D spaces
and information related to a corresponding plurality of 3D space-specific light patterns (See at least Para [0083] “The color pattern 107 definitions and associated functions may be retrieved at operation 401 by the color pattern interface 501 and stored in the pattern-function table 503...”),
wherein, when information sensed by the sensor corresponds to the information in the memory, the emitted two or more light patterns is the two or more predefined light patterns (See at least Para [0083] “The color pattern 107 definitions and associated functions may be retrieved at operation 401 by the color pattern interface 501 and stored in the pattern-function table 503. As will be described in greater detail below, the pattern-function table 503 may be periodically queried to determine 1) a corresponding color pattern 107 that matches measured sensor data and 2) a function corresponding to the detected/matched color pattern 107. These functions may thereafter be used for controlling the robotic device 103.”, Para [0042] “As noted above, in some embodiments, the color sensor 305A and the light sensors 305B may also include a light emitting element 307 (e.g., a light emitting diode (LED)). The light emitting element 307 may be proximate to a corresponding sensor 305 and may be used while the robotic device 103 is operating on a non-digital board 101. For example, the robotic device 103 may detect an absence of light while operating on a non-digital board 101 (e.g., a piece of paper) and in response toggle the light emitting element 307 “ON” to illuminate an area proximate to the corresponding sensor 305. Light reflected off of this non-digital board 101 by each light emitting element 307 may thereafter be detected and analyzed by the sensors 305 to determine intensity of the reflected light. This sensor data may assist in guiding the device 103 along the line 105 and detecting color patterns 107 as will be described in greater detail below.”), and
wherein, when information sensed by the sensor does not correspond to the information in the memory, the two or more light patterns is the two or more random light patterns (See at least Para [0083] “The color pattern 107 definitions and associated functions may be retrieved at operation 401 by the color pattern interface 501 and stored in the pattern-function table 503. As will be described in greater detail below, the pattern-function table 503 may be periodically queried to determine 1) a corresponding color pattern 107 that matches measured sensor data and 2) a function corresponding to the detected/matched color pattern 107. These functions may thereafter be used for controlling the robotic device 103.”, Para [0042] “As noted above, in some embodiments, the color sensor 305A and the light sensors 305B may also include a light emitting element 307 (e.g., a light emitting diode (LED)). The light emitting element 307 may be proximate to a corresponding sensor 305 and may be used while the robotic device 103 is operating on a non-digital board 101. For example, the robotic device 103 may detect an absence of light while operating on a non-digital board 101 (e.g., a piece of paper) and in response toggle the light emitting element 307 “ON” to illuminate an area proximate to the corresponding sensor 305. Light reflected off of this non-digital board 101 by each light emitting element 307 may thereafter be detected and analyzed by the sensors 305 to determine intensity of the reflected light. This sensor data may assist in guiding the device 103 along the line 105 and detecting color patterns 107 as will be described in greater detail below.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Kroyan with Tokunaga and include the feature of storing information that includes information related to the plurality of 3D spaces
and information related to a corresponding plurality of 3D space-specific light patterns, wherein, when information sensed by the sensor corresponds to the information in the memory, the emitted two or more light patterns is the two or more predefined light patterns, and wherein, when information sensed by the sensor does not correspond to the information in the memory, the two or more light patterns is the two or more random light patterns, thereby precise calculation for improved robotic performance in detecting objects (See at least Para [0039] “… This greater resolution may be particularly useful in detecting objects (e.g., intersections along the line 105) as more data points may be available for analysis.”, Para [0047] “…Accordingly, the robotic device 103 may improve performance in bright light conditions by shielding ambient light as described above.”).
Claim(s) 8 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed), and further in view of Brodsky (US 20050058323 A1).
Regarding Claim 8, modified Tokunaga teaches all the elements of claim 1.
However, Tokunaga does not explicitly spell out the robot of claim 4, wherein, when any light pattern, among the two or more random light patterns, causes the object class specific probability of recognition to be higher than or equal to the preset value, the controller stores information related to a certain light pattern in the memory.
Brodsky teaches the robot of claim 4, wherein, when any light pattern, among the two or more random light patterns, causes the object class specific probability of recognition to be higher than or equal to the preset value, the controller stores information related to a certain light pattern in the memory (See at least Claim “3. The traffic monitoring system of claim 1, further comprising a memory for storing prior images from the camera, and wherein the pattern recognizer is further configured to track a path of each of the vehicles based on corresponding headlight patterns in the prior images.”, Para [0025] “Thresholding is a technique that is commonly used to reduce the effects caused by the transient illumination of objects, as illustrated in FIGS. 2A-2B. In FIG. 2B, pixels in the image of FIG. 2A that have a luminance level above a given threshold are given a white value, and pixels having a luminance level below the threshold are given a black value…”, Para [0032] “At 510, an image is received, and at 520, the light patterns within the image are identified. As noted above, thresholding techniques may be used to identify only those light patterns that exceed a given threshold. Pattern matching techniques can also be applied to distinguish headlight patterns, such as recognizing characteristic sizes and shapes of headlight patterns, to distinguish headlights from other vehicle lights as well as from reflections, to further improve the reliability of vehicle identification based on headlight patterns. Thereafter, combinations of headlight patterns can be associated with each vehicle using further conventional pattern matching techniques, including, for example, rules that are based on consistency of movement among patterns, to pair patterns corresponding to a vehicle, as well as rules that are based on the distance between such consistently moving patterns, to distinguish among multiple vehicles traveling at the same rate of speed.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Brodsky and include the feature of when any light pattern, among the two or more random light patterns, causes the object class specific probability of recognition to be higher than or equal to the preset value, the controller stores information related to a certain light pattern in the memory, thereby increase efficiency and reliability by collecting new information for future use which will improve performance of the robot though accurate object recognition (See at least Para [0009] “It is a further object of this invention to facilitate the augmentation of existing video-based traffic monitoring systems to support day and night discrete vehicle identification and tracking.”, Para [0032] “… Pattern matching techniques can also be applied to distinguish headlight patterns, such as recognizing characteristic sizes and shapes of headlight patterns, to distinguish headlights from other vehicle lights as well as from reflections, to further improve the reliability of vehicle identification based on headlight patterns...”, Para [0033] “As is evident from FIG. 2B, however, the high-intensity reflections are often indistinguishable from headlights, and further processing is provided to improve the reliability of the vehicle identification process.”).
Regarding Claim 26, Tokunaga teaches all the elements of claim 16.
However, Tokunaga does not explicitly spell out the robot of claim 16, wherein the controller varies a threshold of an object class probability based on the types of the plurality of objects in the first object recognition mode.
Brodsky teaches the robot of claim 16, wherein the controller varies a threshold of an object class probability based on the types of the plurality of objects in the first object recognition mode (See at least Para [0030] “Track 310 in FIG. 3A illustrates a typical track 310 of a reflection pattern; the end 311 of the track 310 occurring when the reflection is insufficient to exceed a given threshold value. Track 320 in FIG. 3B, on the other hand, illustrates the track of an illumination pattern that exhibits a relatively continuous pattern, having an intensity above the given threshold value for most of the field of view of the camera…”, Para [0032] “At 510, an image is received, and at 520, the light patterns within the image are identified. As noted above, thresholding techniques may be used to identify only those light patterns that exceed a given threshold. Pattern matching techniques can also be applied to distinguish headlight patterns, such as recognizing characteristic sizes and shapes of headlight patterns, to distinguish headlights from other vehicle lights as well as from reflections, to further improve the reliability of vehicle identification based on headlight patterns…”, Para [0025] “Thresholding is a technique that is commonly used to reduce the effects caused by the transient illumination of objects, as illustrated in FIGS. 2A-2B. In FIG. 2B, pixels in the image of FIG. 2A that have a luminance level above a given threshold are given a white value, and pixels having a luminance level below the threshold are given a black value…”, Para [0032] “At 510, an image is received, and at 520, the light patterns within the image are identified. As noted above, thresholding techniques may be used to identify only those light patterns that exceed a given threshold. Pattern matching techniques can also be applied to distinguish headlight patterns, such as recognizing characteristic sizes and shapes of headlight patterns, to distinguish headlights from other vehicle lights as well as from reflections, to further improve the reliability of vehicle identification based on headlight patterns. Thereafter, combinations of headlight patterns can be associated with each vehicle using further conventional pattern matching techniques, including, for example, rules that are based on consistency of movement among patterns, to pair patterns corresponding to a vehicle, as well as rules that are based on the distance between such consistently moving patterns, to distinguish among multiple vehicles traveling at the same rate of speed.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Brodsky and include the feature of varying a threshold of an object class probability based on the types of the plurality of objects in the first object recognition mode, thereby providing distinguishment while detecting objects in order to facilitate light irradiation accordingly which will increase efficiency and reliability (See at least Para [0009] “It is a further object of this invention to facilitate the augmentation of existing video-based traffic monitoring systems to support day and night discrete vehicle identification and tracking.”, Para [0032] “… Pattern matching techniques can also be applied to distinguish headlight patterns, such as recognizing characteristic sizes and shapes of headlight patterns, to distinguish headlights from other vehicle lights as well as from reflections, to further improve the reliability of vehicle identification based on headlight patterns...”, Para [0033] “As is evident from FIG. 2B, however, the high-intensity reflections are often indistinguishable from headlights, and further processing is provided to improve the reliability of the vehicle identification process.”).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed), Goel et al. (US 12434709 B1) (Hereinafter Goel), and further in view of Siegwart et al. (Roland Siegwart; Illah Reza Nourbakhsh; Davide Scaramuzza, "Perception," in Introduction to Autonomous Mobile Robots , MIT Press, 2011, pp.101-263.) (Hereinafter Siegwart).
Regarding Claim 11, modified Tokunaga teaches all the elements of claim 1.
However, Tokunaga does not explicitly spell out the robot of claim 1, wherein the space-specific object recognition process further comprises:
sensing a motion of the object,
predicting a future area where the object is to be located after a predetermined time,
emitting light from the one of the plurality of separately-controllable light-emitting elements so as to track the motion of the object while maintaining the object class specific probability of recognition to be higher than or equal to the preset value.
Goel teaches all the elements of claim 1, wherein the space-specific object recognition process further comprises:
sensing a motion of the object (See at least Col 12 Lines 4-26 “The perception component 322 may include functionality to perform object detection, segmentation, and/or classification. In some examples, the perception component 322 and/or the machine learning component 332 may provide processed sensor data that indicates a presence of an entity that is proximate to the vehicle 302 and/or a classification of the entity as an entity type (e.g., car, pedestrian, cyclist, building, tree, road surface, curb, sidewalk, unknown, etc.). In additional and/or alternative examples, the perception component 322 and/or the machine learning component 332 may provide processed sensor data that indicates one or more characteristics associated with a detected entity and/or the environment in which the entity is positioned. In some examples, characteristics associated with an entity may include, but are not limited to, an x-position (global position), a y-position (global position), a z-position (global position), an orientation, an entity type (e.g., a classification), a velocity of the entity, an extent of the entity (size), etc. Characteristics associated with the environment may include, but are not limited to, a presence of another entity in the environment, a state of another entity in the environment, a time of day, a day of a week, a season, a weather condition, an indication of darkness/light, etc.”),
predicting a future area where the object is to be located after a predetermined time (See at least Col 13 Lines 11-13 “The prediction component 324 may generate one or more probability maps representing prediction probabilities of possible locations of one or more objects in an environment…”), …
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Goel and include the feature of the controller sensing a motion of the object existing in the space through the sensing unit and predicts an area where the object is to be located after a predetermined time based on the sensed motion, thereby taking into account the future positions of obstacles into the calculation making adjustment to precise, accurate, and safe movement of the robot (See at least Col 4 Lines 60-66 “The techniques described herein may improve the functioning of a computing device by providing a robust method of determining changes in environmental conditions associated with environments in which a vehicle is operating, and adjusting vehicle states, adjusting model outputs, and/or selecting model outputs to adjust for changes in environmental conditions. ”).
Siegwart teaches emitting light from the one of the plurality of separately-controllable light-emitting elements so as to track the motion of the object while maintaining the object class specific probability of recognition to be higher than or equal to the preset value (See at least Page 213 Para 5 “Localization accuracy: the detected features should be accurately localized, both in image position and scale. Accuracy is especially important in camera calibration, 3D reconstruction from images (“structure from motion”), and panorama stitching.”, Page 244 Para 3 “Furthermore, we assume that each random variable is subject to a Gaussian probability density curve, with a mean at the true value and with some specified variance:”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Siegwart and include the feature of controlling the lighting-emitting device to emit light so as to track the motion of the one or more objects so as to enable the sensor to continue to recognize the one or more obiects based the object class specific probability of recognition being higher than or equal to the preset value, thereby precisely recognizing one or more objects (See at least Page 143 Para 4 “Only over time, as the underlying performance of imaging chips improves, will significantly more robust vision sensors for mobile robots be available”).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed), and further in view of Mehta et al. (US 20200207375 A1) (Hereinafter Mehta).
Regarding Claim 13, Shadeed teaches all the elements of claim 1.
However, Shadeed does not explicitly spell out the robot of claim 1, wherein the space- specific object recognition process further comprises increasing the preset value of the object class specific probability of recognition when the robot is in a stationary state, and decreasing the preset value of the object class specific probability of recognition when the robot moves.
Mehta teaches the robot of claim 1, wherein the space- specific object recognition process further comprises increasing the preset value of the object class specific probability of recognition when the robot is in a stationary state, and decreasing the preset value of the object class specific probability of recognition when the robot moves (See at least Para [0134] “… In some non-limiting embodiments or aspects, the cost associated with the cost function increases and/or decreases based on autonomous vehicle 104 deviating from a motion plan (e.g., a selected motion plan, an optimized motion plan, a preferred motion plan, etc.). For example, the cost associated with the cost function increases and/or decreases based on autonomous vehicle 104 deviating from the motion plan to avoid a collision with an object.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Mehta and include the feature of the space- specific object recognition process increasing the preset value when the robot is in a stationary state without movement while decreasing the preset value when the robot moves, thereby provide accurate calculation for object recognition and classification which will lead to improved and safe robot navigation.
Claim(s) 24 is rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed), and further in view of Fukuda et al. (JP 2000047296 A) (Hereinafter Fukuda).
Regarding Claim 24, Tokunaga teaches all the elements of claim 16.
However, Tokunaga does not explicitly spell out the robot of claim 16, wherein the controller, in the first object recognition mode, controls the light-emitting device based on an object class probability of a type with a highest priority, with respect to the different types of objects.
Fukuda teaches the robot of claim 16, wherein the controller, in the first object recognition mode, controls the light-emitting device based on an object class probability of a type with a highest priority, with respect to the different types of objects (See at least Para [0006] “According to another aspect of the present invention, there is provided an image recognition apparatus, which comprises a camera for capturing an image of an object and a recognition means for recognizing the object based on image data acquired by the camera. An illumination unit that irradiates the object with light, a storage unit that stores a plurality of illumination conditions of the illumination unit in association with a use priority, and a highest priority among the plurality of illumination conditions. Illumination control means for controlling the illumination means based on the illumination conditions, and when a recognition error by the recognition means occurs, the illumination conditions are changed based on the priority order to cause the camera to re-capture an image…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Fukuda and include the feature of the controller in the first object recognition mode controls the light-emitting device based on an object class probability of a type with a highest priority, with respect to the different types of objects, thereby provide optimal control of the lighting-emitting device in order for precise object recognition and fast robot operation (See at least Para [0005] “Therefore, an object of the present invention is to provide an image recognition apparatus and an image recognition method which are excellent in operability and can improve the operation rate of the apparatus.”).
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed), Fukuda et al. (JP 2000047296 A) (Hereinafter Fukuda), and further in view of Brodsky (US 20050058323 A1).
Regarding Claim 25, Tokunaga teaches all the elements of claim 24.
However, Tokunaga does not explicitly spell out the robot of claim 24, wherein the controller
controls the light-emitting device to emit a light pattern that causes the object class probability of the type with the highest priority to exceeds a type-specific threshold.
Fukuda teaches the robot of claim 24, wherein the controller controls the light-emitting device to emit a light pattern that causes the object class probability of the type with the highest priority to exceeds a type-specific threshold (See at least Para [0006] “According to another aspect of the present invention, there is provided an image recognition apparatus, which comprises a camera for capturing an image of an object and a recognition means for recognizing the object based on image data acquired by the camera. An illumination unit that irradiates the object with light, a storage unit that stores a plurality of illumination conditions of the illumination unit in association with a use priority, and a highest priority among the plurality of illumination conditions. Illumination control means for controlling the illumination means based on the illumination conditions, and when a recognition error by the recognition means occurs, the illumination conditions are changed based on the priority order to cause the camera to re-capture an image…”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the Apparatus of Tokunaga with the teachings of Fukuda and include the feature of the controller controlling the light-emitting device to emit a light pattern that causes the object class probability of the type with the highest priority, thereby provide optimal control of the light-emitting device in order for precise object recognition and fast robot operation (See at least Para [0005] “Therefore, an object of the present invention is to provide an image recognition apparatus and an image recognition method which are excellent in operability and can improve the operation rate of the apparatus.”).
Brodsky teaches … object class probability of the type with the highest priority exceeds the type-specific threshold (See at least Para [0030] “Track 310 in FIG. 3A illustrates a typical track 310 of a reflection pattern; the end 311 of the track 310 occurring when the reflection is insufficient to exceed a given threshold value. Track 320 in FIG. 3B, on the other hand, illustrates the track of an illumination pattern that exhibits a relatively continuous pattern, having an intensity above the given threshold value for most of the field of view of the camera…”, Para [0032] “At 510, an image is received, and at 520, the light patterns within the image are identified. As noted above, thresholding techniques may be used to identify only those light patterns that exceed a given threshold. Pattern matching techniques can also be applied to distinguish headlight patterns, such as recognizing characteristic sizes and shapes of headlight patterns, to distinguish headlights from other vehicle lights as well as from reflections, to further improve the reliability of vehicle identification based on headlight patterns…”, Para [0025] “Thresholding is a technique that is commonly used to reduce the effects caused by the transient illumination of objects, as illustrated in FIGS. 2A-2B. In FIG. 2B, pixels in the image of FIG. 2A that have a luminance level above a given threshold are given a white value, and pixels having a luminance level below the threshold are given a black value…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the system of Shadeed with the teachings of Brodsky and include the feature of object class probability of the type with the highest priority exceeds the type-specific threshold, thereby providing optimal adaptive light irradiation for object recognition which will increase performance accuracy, efficiency and reliability of the robot (See at least Para [0009] “It is a further object of this invention to facilitate the augmentation of existing video-based traffic monitoring systems to support day and night discrete vehicle identification and tracking.”, Para [0032] “… Pattern matching techniques can also be applied to distinguish headlight patterns, such as recognizing characteristic sizes and shapes of headlight patterns, to distinguish headlights from other vehicle lights as well as from reflections, to further improve the reliability of vehicle identification based on headlight patterns...”, Para [0033] “As is evident from FIG. 2B, however, the high-intensity reflections are often indistinguishable from headlights, and further processing is provided to improve the reliability of the vehicle identification process.”).
Claim(s) 27 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Tokunaga (US 20230326056 A1) in view of Shadeed et al (H. Shadeed, J. Wallaschek and S. Mojrzisch, "On Intelligent Adaptive Vehicle Front-Lighting Assistance Systems," 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 2007, pp. 503-507) (Hereinafter Shadeed), Brodsky (US 20050058323 A1), and further in view of Liang et al. (US 20220327314 A1) (Hereinafter Liang).
Regarding Claim 27, Tokunaga teaches all the elements of claim 26.
However, Tokunaga does not explicitly spell out the robot of claim 26, wherein the controller sets the threshold for recognizing the objects to a first threshold, and when a preset type of object is included in the plurality of objects, sets the threshold to a second threshold higher than the first threshold.
Liang teaches the robot of claim 26, wherein the controller sets the threshold for recognizing the objects to a first threshold, and when a preset type of object is included in the plurality of objects, sets the threshold to a second threshold higher than the first threshold (See at least Para [0036] “In a particular example, the video analytics engine 102 may be used to perform object recognition using any suitable technique, and assign a confidence score when determining whether, or not an object in an image 112 comprises a given object. Such a confidence score may be compared to an object recognition confidence threshold to determine whether, or not, the object in an image 112 comprises the given object. In one particular example, the camera 104 may be monitoring an indoor location where vehicles are not “normally” located; as such, an object recognition confidence threshold for detecting a vehicle at such an indoor location may be set relatively high for detecting a vehicle, but relatively low for detecting humans.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Liang and include the feature of controller setting the threshold for recognizing the objects to a first threshold, and when a preset type of object is included in the plurality of objects, sets the threshold to a second threshold higher than the first threshold, thereby providing option of illuminating light according to specific situation which will help detect object more precisely creating an accurate, efficient, and safe robot movement (See at least Para [0014] “…Regardless, such pruning may generally cause the video analytics engine to operate more efficiently as the number of video analytics parameters used to analyze the images by the video analytics engine is reduced.”).
Regarding Claim 28, modified Tokunaga teaches all the elements of claim 27. Tokunaga further teaches the robot of claim 27, wherein the controller, when the preset type of object is included in the plurality of objects, controls the light-emitting device to irradiate a light pattern (See at least (See at least Para [0068] “Furthermore, in order to estimate the three-dimensional shape of the object surface, it is necessary to add depth information at each of a plurality of times…”, Para [0101] “…Furthermore, the information processing apparatus 10 controls pattern irradiation by the irradiator 40...”, Para [0109] “The irradiation control unit 131 controls the irradiator 40 so that the pattern is irradiated by the irradiator 40…”, Para [0006] “an integrated processing unit that estimates three-dimensional information on the basis of the first depth information and position and orientation information of the first sensor at each of a plurality of times.”, Para [0111] “The IR camera 30 captures a first frame (captured image) and outputs the captured first frame to the information processing apparatus 10. The depth information estimating unit 132 detects a part or all of the pattern (a plurality of configuration elements) on the basis of the first frame output from the IR camera 30.”, Para [0193] “Therefore, in the third embodiment of the present disclosure, an irradiation control unit 138 (FIG. 1) controls the irradiator 40 to switch the irradiation pattern among a plurality of mutually different patterns.”, Para [0151] “Note that adjacent surfaces of objects existing in reality are continuous. Therefore, as illustrated in FIG. 8, the complementary processing performed by the integrated processing unit 133 may include processing of estimating the position of the shape corresponding to the disappearance point 81 such that the position of the three-dimensional shape corresponding to the detection point 82 and the position of the three-dimensional shape corresponding to the disappearance point 81 are continuous.”)…
However, Tokunaga does not explicitly spell out … that causes the object class probability is higher than the second threshold.
Liang teaches … that causes the object class probability to be higher than the second threshold (See at least Para [0036] “In a particular example, the video analytics engine 102 may be used to perform object recognition using any suitable technique, and assign a confidence score when determining whether, or not an object in an image 112 comprises a given object. Such a confidence score may be compared to an object recognition confidence threshold to determine whether, or not, the object in an image 112 comprises the given object. In one particular example, the camera 104 may be monitoring an indoor location where vehicles are not “normally” located; as such, an object recognition confidence threshold for detecting a vehicle at such an indoor location may be set relatively high for detecting a vehicle, but relatively low for detecting humans.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the apparatus of Tokunaga with the teachings of Liang and include the feature of causing the object class probability is higher than the second threshold, thereby providing option of illuminating light according to specific situation which will help detect object more precisely creating an accurate, efficient, and safe robot movement (See at least Para [0014] “…Regardless, such pruning may generally cause the video analytics engine to operate more efficiently as the number of video analytics parameters used to analyze the images by the video analytics engine is reduced.”).
Allowable Subject Matter
Claims 14 and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kim et al. (US 20100185328 A) teaches a robot that supplies a projector service according to a
user's context and a controlling method thereof. The robot includes a user detection unit detecting a user; a user recognition unit recognizing the user; an object recognition unit recognizing an object near the user; a position perception unit perceiving relative positions of the object and the user; a context awareness unit perceiving the user's context based on information on the user, the object and the relative positions between the user and the object; and a projector supplying a projector service corresponding to the user's context.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHEDA HOQUE whose telephone number is (571)270-5310. The examiner can normally be reached Monday-Friday 8:00 am- 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached at 571-270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHEDA HOQUE/ Examiner, Art Unit 3658
/Ramon A. Mercado/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights