Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The United States Patent & Trademark Office appreciates the application that is submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below.
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of CN202210933165.9, filed in China on 08/04/2022, and PCT/CN2023/110258, filed in China on 07/31/2023.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 01/06/2026 and 07/07/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Preliminary Amendment
Applicant submitted a preliminary amendment on 01/30/2025. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Status of Claims
Claims 1-10 and 14-18 are pending.
Claims 11-13 are cancelled.
Specification
Applicant is reminded of the proper content of an abstract of the disclosure.
A patent abstract is a concise statement of the technical disclosure of the patent and should include that which is new in the art to which the invention pertains. The abstract should not refer to purported merits or speculative applications of the invention and should not compare the invention with the prior art.
If the patent is of a basic nature, the entire technical disclosure may be new in the art, and the abstract should be directed to the entire disclosure. If the patent is in the nature of an improvement in an old apparatus, process, product, or composition, the abstract should include the technical disclosure of the improvement. The abstract should also mention by way of example any preferred modifications or alternatives.
Where applicable, the abstract should include the following: (1) if a machine or apparatus, its organization and operation; (2) if an article, its method of making; (3) if a chemical compound, its identity and use; (4) if a mixture, its ingredients; (5) if a process, the steps.
Extensive mechanical and design details of an apparatus should not be included in the abstract. The abstract should be in narrative form and generally limited to a single paragraph within the range of 50 to 150 words in length.
See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
The abstract of the disclosure is objected to because the abstract is 189 words. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are: “ motion information acquisition device ” in claim 3 and “ rotating mechanism ” in claims 6, 7, and 15.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Objections
Claim 1 is objected to because of the following informalities:
In Claim 1, line 2, “ liner light ” is suggested to read “ linear light ”.
In Claim 1, line 4, “ liner light ” is suggested to read “ linear light ”.
In Claim 15, line 5, “ from to the housing” is suggested to read “ from the housing ”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-10 and 14-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. The Examiner strongly suggested that appropriate corrections be made to clarify the claim scope.
With respect to Claim 1, the claim recites the following, each of which renders the claim indefinite:
“ the object ” on lines 4 and 8, (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the object” recited on lines 4 and 8 is the same or different from “an imaging object” recited on line 2.
“ the surface depth information ” on line 8 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “surface depth information” in claim 1.
“ the positions of pixels ” in lines 8-9, (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “positions of pixels” in claim 1.
“ the relative motion information ” in line 10, (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “relative motion information” in claim 1.
Claims 2-10 are also rejected under 35 U.S.C. 112(b) by virtue of their dependence on Claim 1.
With respect to Claim 2, the claim recites the following, each of which renders the claim indefinite:
“ the motion information ” in lines 4-5 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the motion information” recited on line 2 of claim 2 is the same or different from “the relative motion information” recited in line 10 of claim 1.
“ the object” in lines 2, 5, 6, and 8 (unclear antecedent basis); it is unclear as to whether “the object” recited in lines 2, 5, 6, and 8 is the same or different from “the imaging object” recited in line 9 of claim 2.
With respect to Claim 4, the claim recites the following, each of which renders the claim indefinite:
“ the depth information ” in line 4 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the depth information” recited on line 4 of claim 4 is the same or different from “the surface depth information” recited in line 8 of claim 1.
With respect to Claim 7, the claim recites the following, each of which renders the claim indefinite:
“ the imaging area ” on lines 2-3, (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of a/an “imaging area”.
“ the rotating mechanism ” on line 3 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. Claim 6 recites “a rotating mechanism,” however, claim 7 does not depend on claim 6, it depends on claim 1, which does not mention “a rotating mechanism.”
“ the length direction ” on line 3, (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of a “length direction”.
With respect to Claim 14, the claim recites the following, each of which renders the claim indefinite:
“ the object “ on line 3 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the object” recited on line 3 is the same or different from “an imaging object” recited in lines 2 and 6 of claim 14.
“ the surface depth information ” on line 6 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “surface depth information” in the claim.
“ the positions of pixels ” on lines 6-7, (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “positions of pixels” in the claim.
“ the relative motion information ” on line 8, (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “relative motion information” in the claim.
Claims 15-18 are also rejected under 35 U.S.C. 112(b) by virtue of their dependence on Claim 14.
With respect to Claim 15, the claim recites the following, each of which renders the claim indefinite:
“ the housing ” in line 5 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a housing” of a depth data measurement device.
With respect to Claim 16, the claim recites the following, each of which renders the claim indefinite:
“ the item ” on lines 1, 8, 10, and 11 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of a/an “item.”
“ the motion information ” on lines 3 and 11 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the motion information” recited on lines 3 and 11 of claim 16 is the same or different from “the relative motion information” recited in line 8 of claim 14.
“ the depth information ” on lines 8 and 10 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the depth information” recited on lines 8 and 10 of claim 16 is the same or different from “the surface depth information” recited in line 6 of claim 14.
“ the volume information ” on line 10 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “volume information”.
With respect to Claim 17, the claim recites the following, each of which renders the claim indefinite:
“ the motion information ” on lines 2 and 11 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the motion information” recited on lines 2 and 11 of claim 17 is the same or different from “the relative motion information” recited on line 8 of claim 14.
“ the depth data measurement device ” on line 2 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a depth data measurement device” in claim 17 or in the claim from which claim 17 depends on (claim 14).
“ the depth information ” on lines 7 and 10 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether “the depth information” recited on lines 7 and 10 of claim 17 is the same or different from “the surface depth information” recited on line 4 of claim 17.
“ the surface ” on lines 7 and 10 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a surface”.
“ the status ” on line 9 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a status”.
“ the surface position ” on line 9 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a surface position”.
With respect to Claim 18, the claim recites the following, each of which renders the claim indefinite:
“ the predetermined depth distribution pattern ” in line 6 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a predetermined depth distribution pattern”.
“ the surface status ” in lines 6 and 10 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a surface status”.
“the actual surface depth distribution ” in line 8 (unclear antecedent basis); there is insufficient antecedent basis for this limitation in the claim. There is no previous mention of “a actual surface depth distribution”.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Taamazyan et al. (U.S. Patent Pub. No. 20230084807, hereafter referred to as Taamazyan) in view of Mishra et al. (U.S. Patent No. 10,282,902, hereafter referred to as Mishra).
Regarding Claim 1, Taamazyan teaches a depth data measurement device (Paragraphs [0087], [0046], Fig. 1, Taamazyan teaches systems for performing 3-D reconstruction using event cameras, thereby enabling generation of depth maps (e.g., 3D models and/or point clouds) of scenes. Fig. 1 (below) is a block diagram of an active scanning system.), comprising: a projection assembly for projecting a liner light onto an imaging object (Paragraphs [0046], [0054], Fig. 1, Taamazyan teaches a projection system (10) which projects or emits structured light onto a scene (2) in the field of projection (10A). The scene (2) may include various objects. The “structured light” may be emitted as a single stripe of light perpendicular to the baseline between the structured light projector and the camera. The Examiner is interpreting “liner light” to be “linear light” in light of Applicant’s specification (see Paragraph [0006]). Further, the Examiner interprets the “stripe” of light to be a linear light.)
PNG
media_image1.png
527
550
media_image1.png
Greyscale
with relative motion (Paragraph [0091], Fig. 6A, Taamazyan teaches multiple objects (650) resting on the top surface (644) of the conveyor belt, where the conveyor belt (640) conveys/moves the objects (650) along a direction of motion, as indicated by the arrow in Fig. 6A, where the objects (650) are moved into, then out of, the field of view (620A) of the event camera. In the arrangement shown in Fig. 6A, a substantially constant illumination is provided to the scene. The substantially constant illumination may be provided by a projection system similar to the projection system (10). The Examiner interprets since the multiple objects are moving along a conveyor belt, the objects have “relative motion”.);
PNG
media_image2.png
543
948
media_image2.png
Greyscale
an image sensor for imaging the object based on the reflected light of the liner lights (Paragraph [0065], Fig. 1, Taamazyan teaches the projection system (10) and the event camera (20) are arranged such that the event camera (20) is arranged to detect the light projected onto the scene (2) by the projection system (10) after reflecting off surfaces of the scene (2). The Examiner is interpreting “liner light” to be “linear light” in light of Applicant’s specification (see Paragraph [0006]).) the image sensor including a plurality of pixels (Paragraphs [0071], [0073], Fig. 5A (reference character 520A), Fig. 5B (reference character 520B), Taamazyan teaches an 8x6 grid of sensor pixels of the image sensor that receive the light from the portion of the scene (2) corresponding to the sphere is schematically depicted at 520A, where the white pixels indicate locations where illumination was provided by the projection system (10) and dark pixels indicate locations that were not illuminated by the projection system. Also see Fig. 5B (520B) for another example of the 8x6 grid of sensor pixels having a different arrangement.),
PNG
media_image3.png
189
205
media_image3.png
Greyscale
PNG
media_image4.png
187
203
media_image4.png
Greyscale
and each pixel sending a valid signal with a timestamp when the change in the amount of light received per unit time exceeds a threshold (Paragraphs [0043], [0071], [0074], [0077], Fig. 5C, Taamazyan teaches when a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event (“change event”), where the event is timestamped and may indicate the direction of the change in brightness of that pixel (brighter or darker), and in some cases, may indicate a magnitude of that change. A positive change event corresponds to a change at that pixel to a value of 1 due to and/or corresponding to the pattern projected during a particular time period. A negative change event during a particular time period corresponds to a change to a value of 0. No change event for a particular time period corresponds to the pixel having the same brightness level as the previous time period (e.g., a 0 or a 1).);
PNG
media_image5.png
559
775
media_image5.png
Greyscale
and a processor (Paragraph [0006], Taamazyan teaches the controller including a processor and memory, the memory storing instructions that, when executed by the processor, cause the controller to: see next limitation.) for calculating the surface depth information of the object based on the positions of pixels corresponding to the valid signals (Paragraph [0006], Taamazyan teaches computing a plurality of depths of surfaces imaged by the event camera at the event pixels associated with the first change events to generate a depth map. The Examiner interprets “event pixels” to be pixels corresponding to the valid signals (change in light exceeds threshold).), the timestamps of those valid signals (Paragraphs [0043], [0048], Taamazyan teaches an event camera is a type of image capture device that captures the change in brightness at each pixel instead of capturing the actual brightness value at each pixel. When a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event, where the event is timestamped and may indicate the change in brightness at the pixel (e.g., brighter or darker) and in some cases, the magnitude of that change. Pixel-level events include the row and column of the pixel that generated the event (x, y) coordinates of the pixel within the image sensor, whether the pixel-level event was an ON event or an OFF event, and a timestamp of the pixel-level event.),
Taamazyan does not explicitly disclose (calculating the surface depth information of the object based on) the relative motion information.
Mishra is in the same field of art of capturing depth data for objects in motion, such as objects moving on a conveyor belt. Further, Mishra teaches (calculating the surface depth information of the object based on) the relative motion information (Col. 9, lines 8-15 and lines 18-30, Mishra teaches using the known velocity (e.g., direction and speed) of the conveyor and the object in motion thereon, the 3D model (surface model) of the object defined using the first set of imaging data may be projected forward in a direction and by a distance of travel of the object defined by a product of a speed and an elapsed time between the first time and the second time, e.g., by advancing each of the vertices of the point cloud in the direction and by the distance. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets that 3D models (surface models) of the object(s) include “surface depth information” since they are a representation of depths to the object (see Col. 8, lines 38-44). Further, the 3D model (depth information) is determined using known velocity information for the object in motion.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan by using the known velocity (direction and speed) of the object on a conveyor belt to determine depth information for an object that is taught by Mishra, to make the invention that generates surface models (which include depth information) such as meshes, point clouds, etc. for objects in motion on a conveyor belt; thus, one of ordinary skilled in the art would be motivated to combine the references since collecting motion information such as velocity of the object on a conveyor belt enables the creation of simulated 3D models from an earlier time and/or later time by projecting the 3D models (which include depth and texture data) in a direction of travel (forward or backward) since depth data captured at a later time, for example, may be more accurate than depth data captured at an earlier time (Mishra, Col. 19, lines 52-67 through Col. 20, lines 1-2).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 2, Taamazyan in view of Mishra discloses the depth data measurement device according to claim 1, wherein the relative motion is the motion of the object relative to the depth data measurement device (Col. 10, lines 49-52, Col. 2, lines 31-32, Mishra teaches relative movement with respect to an imaging device (e.g., where the object is in motion and the imaging device is stationary). The imaging device can capture depth data.), and: when the depth data measurement device is stationary (Col. 10, lines 49-52, Col. 2, lines 31-32, Mishra teaches the imaging device is stationary. The imaging device can capture depth data.), obtaining the motion information of the object as the relative motion information (Col. 12, line 67 through Col. 3, lines 1-2, Mishra teaches the conveyor includes an item thereon, and operates to advance the item forward at a velocity VCON.); when the object is stationary (Col. 10, lines 52-53, Mishra teaches where the object is stationary and the imaging device is in motion.), obtaining the motion information of the depth data measurement device as the relative motion information (Col. 10, lines 44-57, Col. 2, lines 31-32, Mishra teaches leveraging relative movement of an object with respect to an imaging device. For example, where the object is stationary and the imaging device is in motion. The imaging device can capture depth data.); or when the object and the depth data measurement device are both in motion (Col. 10, lines 44-57, Mishra teaches where the object and the imaging are in motion at different velocities. Under BRI, the Examiner interprets “or” to mean only one of the limitations (either object is stationary or object and depth measurement device are both in motion) to be required. Citation is included for completeness.), obtaining the relative motion information of the imaging object and the depth data measurement device as the relative motion information (Col. 10, lines 44-57, Col. 2, lines 31-32, Mishra teaches leveraging relative movement of an object with respect to an imaging device. For example, where the object and the imaging are in motion at different velocities. The imaging device can capture depth data.).
In regards to Claim 5, Taamazyan in view of Mishra teaches the depth data measurement device according to claim 1, wherein the projection assembly projects linear light pulses at a predetermined frequency (Paragraph [0078], Taamazyan teaches the projection system operates semi-autonomously and projects different patterns onto a scene during different time periods, as controlled by a timer and set of stored patterns or other control of patterns (e.g., a digital counter) internal to the projection system.).
In regards to Claim 14, Taamazyan teaches a depth data measurement method (Paragraph [0032], Fig. 4, Taamazyan teaches a method for computing depth of pixels based on events detected by an event camera.), comprising: projecting a linear light onto an imaging object with relative motion (Paragraphs [0065], [0091], [0054], Fig. 6A, Taamazyan teaches projecting calibration patterns onto a scene. The scene may include imaging objects moving along a conveyor belt, such as depicted in Fig. 6A. A single stripe of light may be emitted, where the stripe of light is perpendicular to the baseline between the structured light projector and the camera.); imaging the object based on reflected light of the linear light (Paragraph [0065], Taamazyan teaches the event camera (20) is arranged to detect light projected onto a scene (2) by the projection system (10) after reflecting off surfaces of the scene (2).), and each pixel sending a valid signal with a timestamp when the change in the amount of light received per unit time exceeds a threshold (Paragraphs [0043], [0071], [0074], [0077], Fig. 5C, Taamazyan teaches when a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event (“change event”), where the event is timestamped and may indicate the direction of the change in brightness of that pixel (brighter or darker), and in some cases, may indicate a magnitude of that change. A positive change event corresponds to a change at that pixel to a value of 1 due to and/or corresponding to the pattern projected during a particular time period. A negative change event during a particular time period corresponds to a change to a value of 0. No change event for a particular time period corresponds to the pixel having the same brightness level as the previous time period (e.g., a 0 or a 1).); and calculating the surface depth information of the imaging object based on the positions of pixels corresponding to the valid signals (Paragraphs [0006], [0079], Taamazyan teaches computing a plurality of depths of surfaces imaged by the event camera at the event pixels associated with the first change events to generate a depth map. The Examiner interprets “event pixels” to be pixels corresponding to the valid signals (change in light exceeds threshold).), the timestamps of those valid signals (Paragraphs [0043], [0048], Taamazyan teaches an event camera is a type of image capture device that captures the change in brightness at each pixel instead of capturing the actual brightness value at each pixel. When a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event, where the event is timestamped and may indicate the change in brightness at the pixel (e.g., brighter or darker) and in some cases, the magnitude of that change. Pixel-level events include the row and column of the pixel that generated the event (x, y) coordinates of the pixel within the image sensor, whether the pixel-level event was an ON event or an OFF event, and a timestamp of the pixel-level event.),
Taamazyan does not explicitly disclose (calculating the surface depth information of the imaging object based on)
Mishra is in the same field of art of capturing depth data of objects in motion, such as objects on a conveyor belt. Further, Mishra teaches (calculating the surface depth information of the object based on) the relative motion information (Col. 9, lines 8-15 and lines 18-30, Mishra teaches using the known velocity (e.g., direction and speed) of the conveyor and the object in motion thereon, the 3D model (surface model) of the object defined using the first set of imaging data may be projected forward in a direction and by a distance of travel of the object defined by a product of a speed and an elapsed time between the first time and the second time, e.g., by advancing each of the vertices of the point cloud in the direction and by the distance. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets that 3D models (surface models) of the object(s) include “surface depth information” since they are a representation of depths to the object (see Col. 8, lines 38-44). Further, the 3D model (depth information) is determined using known velocity information for the object in motion.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan by using the known velocity (direction and speed) of the object on a conveyor belt to determine depth information for an object that is taught by Mishra, to make the invention that generates surface models (which include depth information) such as meshes, point clouds, etc. for objects in motion; thus, one of ordinary skilled in the art would be motivated to combine the references since since collecting motion information such as velocity of the object on a conveyor belt enables the creation of simulated 3D models from an earlier time and/or later time by projecting the 3D models (which include depth and texture data) in a direction of travel (forward or backward) since depth data captured at a later time, for example, may be more accurate than depth data captured at an earlier time (Mishra, Col. 19, lines 52-67 through Col. 20, lines 1-2).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 3 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over Taamazyan et al. (U.S. Patent Pub. No. 20230084807, hereafter referred to as Taamazyan) in view of Mishra et al. (U.S. Patent No. 10,282,902, hereafter referred to as Mishra) in further view of Zhang et al. (U.S. Patent No. 11,335,018 hereafter referred to as Zhang).
Regarding Claim 3, Taamazyan in view of Mishra teaches the depth data measurement device according to claim 2.
Taamazyan in view of Mishra does not explicitly disclose wherein the depth data measurement device further includes: a motion information acquisition device for acquiring the relative motion information.
Zhang is in the same field of art of acquiring depth information for an object in motion, such as an object on a conveyor belt. Further, Zhang teaches wherein the depth data measurement device further includes: a motion information acquisition device for acquiring the relative motion information (Col. 12, lines 23-26, Zhang teaches a determination module for determining traversal speed.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan in view of Mishra by acquiring the speed of the conveyor belt that is taught by Zhang, to make the invention that acquires the speed of the conveyor belt on which the object is sitting on in order to calculate depth information/volume information; thus, one of ordinary skilled in the art would be motivated to combine the references since the conveyor belt may have various speeds, for example, the conveyor belt may be set manually or automatically adjusted according to varying situations, such as time and amount of packages/objects and the speed is required in order to calculate the volume information of the object/package (Zhang, Col. 1, lines 35-60).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 16, Taamazyan in view of Mishra teaches the method according to claim 14, wherein the object is the item on a conveyor belt (Paragraphs [0089-92], Fig. 6A, Taamazyan teaches multiple objects (650) resting on the top surface (644) of the conveyor belt.), and further includes: (Paragraph [0006], Taamazyan teaches computing a plurality of depths of surfaces imaged by the event camera at the event pixels associated with the first change events to generate a depth map. The Examiner interprets “event pixels” to be pixels corresponding to the valid signals (change in light exceeds threshold).), the timestamps of those valid signals (Paragraphs [0043], [0048], Taamazyan teaches an event camera is a type of image capture device that captures the change in brightness at each pixel instead of capturing the actual brightness value at each pixel. When a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event, where the event is timestamped and may indicate the change in brightness at the pixel (e.g., brighter or darker) and in some cases, the magnitude of that change. Pixel-level events include the row and column of the pixel that generated the event (x, y) coordinates of the pixel within the image sensor, whether the pixel-level event was an ON event or an OFF event, and a timestamp of the pixel-level event.), as well as the relative motion information (Col. 9, lines 8-15 and lines 18-30, Mishra teaches using the known velocity (e.g., direction and speed) of the conveyor and the object in motion thereon, the 3D model (surface model) of the object defined using the first set of imaging data may be projected forward in a direction and by a distance of travel of the object defined by a product of a speed and an elapsed time between the first time and the second time, e.g., by advancing each of the vertices of the point cloud in the direction and by the distance. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets that 3D models (surface models) of the object(s) include “surface depth information” since they are a representation of depths to the object (see Col. 8, lines 38-44).) further includes: calculating the depth information of the item at the corresponding timestamps based on the pixel positions corresponding to the valid signals (Paragraphs [0006], [0043], Taamazyan teaches computing a plurality of depths of surfaces imaged by the event camera at the event pixels associated with the first change events to generate a depth map. Each pixel associated with a change event is timestamped.);
Taamazyan in view of Mishra does not explicitly disclose obtaining the motion information of the conveyor belt as the relative motion information, and calculating the volume information of the item based on the depth information of the item at multiple timestamps and the motion information of the item.
Zhang is in the same field of art of acquiring depth information for an object in motion, such as for an object on a moving conveyor belt. Further, Zhang teaches obtaining the motion information of the conveyor belt as the relative motion information (Col. 7, lines 61-67 through Col. 8, lines 1-3, Zhang teaches determining the initial speed and traversal accuracy according to a speed of the conveyor belt and a required accuracy. For example, the speed of the conveyor belt is determined to be equal to 0.5 m/s.), and calculating the volume information of the item (Col. 8, lines 14-23, Zhang teaches calculating a package volume.) based on the depth information of the item at multiple timestamps (Abstract, Col. 3, lines 1-2, Zhang teaches matching a first depth value of a current pixel point with a second depth value of a target pixel point corresponding to the current pixel point. A time difference (t) is acquired between the first depth image and the second depth image. The Examiner interprets since there is a “time difference” between the first and second depth images/values that they are taken at “multiple timestamps.”) and the motion information of the item (Col. 8, lines 14-23, Zhang teaches calculating a package volume according to the instantaneous speed. The package volume is calculated by the flow volume calculation algorithm based on the fixed speed.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan in view of Mishra by using the relative motion/speed information of the object and depth information of the object to calculate the volume of the object in motion that is taught by Zhang, to make the invention that automatically calculates the volume of an object such as a package using data collected from sensors (depth sensors, etc.) while the package is in motion; thus, one of ordinary skilled in the art would be motivated to combine the references since the automatic calculation of the package volume can free staff from heavy volumetric work (preventing manual measurement using tape measurer), so the labor force is saved, the working efficiency of logistics enterprises is improved, thereby playing an important role in promoting the modernization of the logistics enterprises (Zhang, Col. 1, lines 18-60).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Taamazyan et al. (U.S. Patent Pub. No. 20230084807, hereafter referred to as Taamazyan) in view of Mishra et al. (U.S. Patent No. 10,282,902, hereafter referred to as Mishra) in further view of Chen et al. (U.S. Patent Pub. No. 20120262553, hereafter referred to as Chen).
Regarding Claim 4, Taamazyan in view of Mishra discloses the depth data measurement device according to claim 1, wherein the image sensor includes: a first image sensor (Paragraph [0014], Taamazyan teaches an event camera including a plurality of event pixels.) and a second image sensor (Paragraph [0011], Taamazyan teaches the active scanning system may further include a second event camera forming a stereo pair with the event camera.) (Paragraph [0006], Taamazyan teaches the controller including a processor and memory, the memory storing instructions that, when executed by the processor, cause the controller to perform a method.) calculates the depth information of the object using the positions of pixels corresponding to the valid signals (Paragraph [0006], Taamazyan teaches computing a plurality of depths of surfaces imaged by the event camera at the event pixels associated with the first change events to generate a depth map. The Examiner interprets “event pixels” to be pixels corresponding to the valid signals (change in light exceeds threshold).) and the timestamps of those valid signals (Paragraphs [0043], [0048], Taamazyan teaches an event camera is a type of image capture device that captures the change in brightness at each pixel instead of capturing the actual brightness value at each pixel. When a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event, where the event is timestamped and may indicate the change in brightness at the pixel (e.g., brighter or darker) and in some cases, the magnitude of that change. Pixel-level events include the row and column of the pixel that generated the event (x, y) coordinates of the pixel within the image sensor, whether the pixel-level event was an ON event or an OFF event, and a timestamp of the pixel-level event.) from the first image sensor and the second image sensor (Paragraphs [0083-84], Taamazyan teaches the multiple event cameras may be arranged as one or more stereo pairs, where a stereo pair of event cameras are calibrated with respect to one another and have substantially parallel optical axes with overlapping fields of view to image a scene from different viewpoints. The depth of an imaged surface can be determined based on a disparity calculation (e.g., detecting the difference in position of the detected local portion of the light pattern along the epipolar line between the event cameras.), as well as the relative motion information (Col. 9, lines 8-15 and lines 18-30, Col. 14, lines 34-41, Mishra teaches using the known velocity (e.g., direction and speed) of the conveyor and the object in motion thereon, the 3D model (surface model) of the object defined using the first set of imaging data may be projected forward in a direction and by a distance of travel of the object defined by a product of a speed and an elapsed time between the first time and the second time, e.g., by advancing each of the vertices of the point cloud in the direction and by the distance. Two or more imaging devices may be provided, each including imaging devices having both a color sensor and a depth sensor. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets that 3D models (surface models) of the object(s) include “surface depth information” since they are a representation of depths to the object (see Col. 8, lines 38-44).).
Taamazyan in view of Mishra does not explicitly disclose (the first and second image sensors) disposed on both sides of the projection assembly.
Chen is in the same field of art of acquiring depth information using a projection device and one or more imagers. Further, Chen teaches (the first and second image sensors) disposed on both sides of the projection assembly (Paragraphs [0040], [0091], Fig. 1, Fig. 9A, Chen teaches a depth image acquiring device which includes one projecting device (110), and at least two image sensing devices (120) and (130). The image sensing devices (120) and (130) mainly acquire images of the projection pattern after being projected to the object, so as to obtain image information. See Figures below. The two image sensing devices are on both sides of the projecting device.).
PNG
media_image6.png
331
479
media_image6.png
Greyscale
PNG
media_image7.png
265
446
media_image7.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan in view of Mishra by using two imagers/cameras to acquire images of the object from multiple observation points/perspectives that is taught by Chen, to make the invention that determines depth information using corresponding points of two images captured by two cameras (in different positions); thus, one of ordinary skilled in the art would be motivated to combine the references since by calculating a spatial corresponding relation between each image sensing device and the projecting device, the problem of image occlusion can be mitigated and accuracy may be improved (Chen, Paragraph [0039]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 6 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over Taamazyan et al. (U.S. Patent Pub. No. 20230084807, hereafter referred to as Taamazyan) in view of Mishra et al. (U.S. Patent No. 10,282,902, hereafter referred to as Mishra) in further view of Wang et al. (U.S. Patent Pub. No. 20190045151, hereafter referred to as Wang).
Regarding Claim 6, Taamazyan in view of Mishra discloses the depth data measurement device according to claim 1.
Taamazyan in view of Mishra does not explicitly disclose wherein the relative motion is the motion of the linear light projected by the projection assembly relative to the image sensor and the depth data measurement device, and the projection assembly performs scanning projection of the linear light by rotating a rotating mechanism.
Wang is in the same field of art of capturing depth information for a 3D object and using pixel-specific timestamping. Further, Wang teaches wherein the relative motion is the motion of the linear light projected by the projection assembly relative to the image sensor and the depth data measurement device (Paragraphs [0044], [0051], Wang teaches the laser (33) may be rotatable in X-Y directions. The laser may have angular motions in the X- and Y-directions.), and the projection assembly performs scanning projection of the linear light by rotating a rotating mechanism (Paragraphs [0044], [0051], Fig. 4, Wang teaches the laser (33) may be fixed in one position within the housing of the device (15), but may be rotatable in X-Y directions. The laser (33) may be X-Y addressable (for example, by the laser controller (34)) to perform point scan of the 3D object. In Fig. 4, the X-Y rotational capabilities of the laser source are illustrated using the arrows 62, 64 depicting the laser’s angular motions in the x-direction (having angle “β”) and in the y-direction (having angle “α”). The laser controller (34) may control the X-Y rotation of the laser source based on scanning instructions/input received from the processor. The Examiner interprets the “laser controller” to be the rotating mechanism since it controls the X-Y rotation of the laser.).
PNG
media_image8.png
485
333
media_image8.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan in view of Mishra by scanning the 3D object by rotating a rotating mechanism to control the projected light source that is taught by Wang, to make the invention that completely scans the entire 3D object; thus, one of ordinary skilled in the art would be motivated to combine the references to initiate performing the 3D depth measurements of the object surface facing the scanner. By enabling rotation along the X-Y plane, the scanner may be able to scan the entire surface of the 3D object (Wang, Paragraph [0051]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 15, Taamazyan in view of Mishra discloses the method according to claim 14, wherein projecting the linear light onto the imaging object further includes: (Col. 10, lines 44-57, Col. 4, lines 55-58, Mishra teaches a relative movement of an object with respect to an imaging device (e.g., where the object is motion and the imaging device is stationary, where the object is stationary and the imaging device is in motion, or where the object and the imaging device are in motion at different velocities). The imaging device may be a depth sensor and/or capture depth imaging data.).
Taamazyan in view of Mishra does not explicitly disclose scanning and projecting the linear light onto the imaging object by rotating a rotating mechanism; and projecting the linear light in a direction that remains unchanged from to the housing of a depth data measurement device.
Wang is in the same field of art of capturing depth information for a 3D object and using pixel-specific timestamping. Further, Wang teaches wherein projecting the linear light onto the imaging object further includes: scanning and projecting the linear light onto the imaging object by rotating a rotating mechanism (Paragraphs [0044], [0051], Fig. 2, Wang teaches the laser (33) may be fixed in one position within the housing of the device (15), but may be rotatable in X-Y directions. The laser (33) may be X-Y addressable (for example, by the laser controller (34)) to perform point scan of the 3D object (26). In Fig. 4, the X-Y rotational capabilities of the laser source are illustrated using the arrows 62, 64 depicting the laser’s angular motions in the x-direction (having angle “β”) and in the y-direction (having angle “α”). The laser controller (34) may control the X-Y rotation of the laser source based on scanning instructions/input received from the processor. The Examiner interprets the “laser controller” to be the rotating mechanism since it controls the X-Y rotation of the laser.); projecting the linear light in a direction that remains unchanged from to the housing of a depth data measurement device (Paragraph [0043], Fig. 2, Wang teaches illuminating the 3D object associated with corresponding dotted lines 30-31 representing an illumination path of a light beam that may be used to point scan the 3D object within an optical field of view. A line-by-line point scan of the object surface may be performed using for example, a laser light source (33) operated and controlled by a laser controller (34). The Examiner interprets the linear light is projected in a direction that remains “unchanged” from the housing since it is projected directly to the 3D object and is not obstructed/deterred from reaching the object (illumination path 30-31 in Fig. 2).).
PNG
media_image9.png
429
483
media_image9.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan in view of Mishra by scanning the 3D object by rotating a rotating mechanism to control the projected light source that is taught by Wang, to make the invention that completely scans the 3D object; thus, one of ordinary skilled in the art would be motivated to combine the references to be able to initiate the 3D depth measurements of the object surface facing the scanner. By enabling rotation along the X-Y plane, the scanner may be able to scan the entire surface of the 3D object (Wang, Paragraph [0051]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 7-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over Taamazyan et al. (U.S. Patent Pub. No. 20230084807, hereafter referred to as Taamazyan) in view of Mishra et al. (U.S. Patent No. 10,282,902, hereafter referred to as Mishra) in further view of Link et al. (U.S. Patent Pub. No. 20190258225, hereafter referred to as Link).
Regarding Claim 7, Taamazyan in view of Mishra discloses the depth data measurement device according to claim 1, wherein the projection assembly scans and projects a linear light (Paragraph [0046], Taamazyan teaches a projection system and an event camera. A controller controls the projection system to project or emit structured light onto a scene in a field of projection and to receive image data from the event camera.) moving along a first direction to the imaging area (Col. 3, lines 40-55, Col. 2, lines 65-67 through Col. 3, lines 1-14, Mishra teaches the imaging device (150) captures imaging data, e.g., depth imaging data, regarding the item (10). The item (10) is advanced at velocity VCON on the conveyor. ) (Paragraphs [0056], Fig. 3A, Taamazyan teaches a sequence of different binary patterns of stripes may be projected onto the scene, where positions within the field of projection are “on” or “off” in different patterns, and where the sequence of “on” and “off” encodes the location of the emitted light within the field of projection.);
PNG
media_image10.png
546
776
media_image10.png
Greyscale
the processor (Paragraph [0006], Taamazyan teaches the controller including a processor and memory, the memory storing instructions that, when executed by the processor, cause the controller to perform a method.) is used to generate a two-dimensional image of the striped light pattern based on brightness of the striped light pattern corresponding to the pixel positions and timestamps corresponding to the valid signals (Paragraphs [0077], [0043], Taamazyan teaches reconstructing the projected structured light codes that were detected as pixels of the event camera based on the camera-level change events received by the event camera. Change events occur when a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event, where the event is timestamped and may indicate the direction of the change in brightness of that pixel.), and obtain the surface depth information of the object in the imaging area according to the two-dimensional image (Paragraph [0079], Taamazyan teaches determining the depths of surfaces in scene (2) as imaged by the event camera based on the reconstructed codes at the locations of the pixels, and by applying the techniques for structured 3D light reconstruction.).
Taamazyan in view of Mishra does not explicitly disclose (wherein the projection assembly scans and projects a linear light moving along a first direction to the imaging area) through the rotation of the rotating mechanism, wherein the length direction of the linear light is a second direction perpendicular to the first direction.
Link is in the same field of art of capturing depth information for object(s) moving along a conveyor belt for inspection of objects. Further, Link teaches (wherein the projection assembly scans and projects a linear light moving along a first direction to the imaging area) through the rotation of the rotating mechanism (Paragraph [0165-166], Fig. 1A, Link teaches the laser module (200) is rotated 90 degrees such that the laser field of view is parallel to direction A (see Fig. 1A). The laser scans a path across the transport. Laser control motors control movement of the laser in the X-Y plane and the Z direction.), wherein the length direction of the linear light is a second direction perpendicular to the first direction (Paragraph [0166], Fig. 14, Link teaches the laser module (200) moves during a scanning process in which the object moves in direction A and the laser module moves only in directions perpendicular to direction A. As shown in Fig. 14, as the laser module moves in directions B and C, perpendicular to direction A, the field of view of the laser scans a path across the transport.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan in view of Mishra by attaching the laser/light projector/emitter to a rotating mechanism to move the light along the object to scan the object that is taught by Link, to make the invention that makes multiple scans of the object (with the laser) during each pass; thus, one of ordinary skilled in the art would be motivated to combine the references since enabling rotation of the scanner allows overlapping scanning of the object, which can reduce inherent noise in translation of both the object on transport and the laser module, which increases the sampling resolution. Having additional data for each point on the point cloud allows for reduction in the total scan error by minimizing data variance (Link, Paragraph [0152]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 8, Taamazyan in view of Mishra in further view of Link discloses the depth data measurement device according to claim 7, wherein the projection assembly completes a pattern scanning projection within a scanning cycle T (Paragraph [0078], Taamazyan teaches the projection system projects different patterns onto a scene during different time periods, as controlled by a timer and a set of stored patterns. The Examiner interprets a “time period” which a projection system projects a pattern onto a scene to be a “scanning cycle.”), and within the scanning cycle T (Paragraph [0071], Fig. 5A, Taamayzan teaches during the first time period (e.g., t1), the controller controls the projection system (10) to emit a first pattern (310A) onto the scene (2). The Examiner interprets the “first time period” to be the scanning cycle.), the projection assembly adjusts the brightness of the linear light in the moving projection according to the single striped light pattern to be projected (Paragraph [0070], Fig. 5A (reference character 310), Taamayzan teaches the controller controls the projection system (10) to project a first pattern onto the scene (2).), and the projection assembly adjusts the brightness of the linear light according to different striped light patterns (Paragraphs [0076], [0072], Fig. 3B, Fig. 3C, Fig. 4, Taamayzan teaches there may be a stored and/or otherwise specified sequence of patterns to be projected onto a scene to perform structured light reconstruction. If there are additional patterns to project, the controller controls the projection system to project the next pattern, and the process loops until all the different patterns of the sequence have been projected.) in N consecutive scanning cycles T (Paragraphs [0061-62], Taamayzan teaches the different binary patterns are projected during different time intervals t1, t2, t3, t4, and t5. The projection system emits different binary patterns 312, 313, 314, and 315 during periods t1, t2, t3, t4, and t5, respectively. The Examiner interprets each time period (t1, t2, and so on) to be “scanning cycles.” In addition, the time intervals are in consecutive order, hence t1, t2, t3… and so on.), the processor (Paragraph [0006], Taamazyan teaches the controller including a processor and memory, the memory storing instructions that, when executed by the processor, cause the controller to perform a method.) generates N two-dimensional images corresponding to N striped light patterns based on the N scanning cycles T (Paragraphs [0055], [0061-62], Taamayzan teaches multiple different patterns may be projected over time by a structured light projector, where the patterns of light are designed such that, it can be determined from the captured images, which portions of the scene are illuminated by particular locations within the field of projection of the structured light projector. The different patterns are projected during different time intervals t1, t2, t3, t4,and t5. The Examiner interprets each time interval to be a “scanning cycle.”), and synthesizes a single depth image of the object in the imaging area according to the N two-dimensional images (Paragraph [0099], Taamayzan teaches a depth map is computed from the events generated by the event cameras during the projection of the one or more patterns.).
In regards to Claim 9, Taamazyan in view of Mishra in further view of Link discloses the depth data measurement device according to claim 7, wherein the projection assembly completes a pattern scan within a scanning cycle T (Paragraph [0078], Taamazyan teaches the projection system projects different patterns onto a scene during different time periods, as controlled by a timer and a set of stored patterns. The Examiner interprets a “time period” which a projection system projects a pattern onto a scene to be a “scanning cycle.”), and within the scanning cycle T (Paragraph [0071], Fig. 5A, Taamayzan teaches during the first time period (e.g., t1), the controller controls the projection system (10) to emit a first pattern (310A) onto the scene (2). The Examiner interprets the “first time period” to be the scanning cycle.), the projection assembly alternately adjusts the brightness of the linear light in the moving projection according to the N striped light patterns to be projected (Paragraphs [0061-62], Figs. 3A-3C, Taamazyan teaches the structured light projector projects different binary patterns (310) during time intervals t1, t2, t3, t4, and t5. Periods in which the binary pattern emits light onto the subject pixels (A, B, C) of the camera are labeled “1” and periods in which the binary pattern does not emit light onto the subject pixels A, B, C are labeled “0”. The Examiner interprets each pattern “alternately adjusts” the brightness of the light in the N striped light patterns since each pattern contains regions with no projected illumination (darker/shaded region) “0” and regions with projected illumination (lighter/non-shaded region) “1”. (see Fig. 3A-3C).),
PNG
media_image11.png
530
784
media_image11.png
Greyscale
and the processor generates N two-dimensional images corresponding to the N striped light patterns (Paragraph [0071], Taamazyan teaches the light reflecting off the sphere in a scene and toward the event camera may form an image (510A) on the image sensor, where the portions of the sphere that are not illuminated by the additional projected light are shown with shading (e.g., where the illuminated area has a white crescent shape surrounding a shaded, rounded area). An 8×6 grid of sensor pixels of the image sensor 22 that receive light from the portion of the scene 2 corresponding to the sphere is schematically depicted at 520A, where the white pixels indicate locations where illumination was provided by the projection system 10 and dark pixels indicate locations that were not illuminated by the projection system.)
PNG
media_image12.png
592
833
media_image12.png
Greyscale
based on the one scanning cycle T (Paragraph [0071], Fig. 5A, Taamazyan teaches during the first time period (t1), the controller controls the projection system to emit a first pattern (310A) onto the scene (2). The Examiner interprets the first time period (t1) to be one scanning cycle.), and synthesizes a single depth image of the object in the imaging area based on the N two-dimensional images (Paragraph [0079], Taamazyan teaches determining the depths of surfaces in the scene (2) as imaged by the event camera (20) based on the reconstructed codes at the locations of the pixels.).
In regards to Claim 10, Taamazyan in view of Mishra in further view of Link discloses the depth data measurement device according to claim 7, wherein the projection assembly completes a scan within a scanning cycle T (Paragraph [0071], Taamazyan teaches a first projected pattern during a first time period (e.g., t1).), and the processor (Paragraph [0006], Taamazyan teaches the controller including a processor and memory, the memory storing instructions that, when executed by the processor, cause the controller to perform method.) assigns a brightness corresponding to a predetermined striped light pattern to the pixel position corresponding to the timestamp (Paragraph [0071], Fig. 5A, (reference character 520A), Taamazyan teaches an 8x6 grid of sensor pixels of the image sensor that receive light from the portion of the scene corresponding to the sphere is depicted in Fig. 5A (reference character 520A), where the white pixels indicate locations where illumination was provided by the projection system (which projects a pattern (310A) such as an alternating pattern of four stripes, where the first and third stripes have no projected light “0”, and the second and fourth stripes have projected light “1”) and dark pixels indicate locations that were not illuminated by the projection system. The event camera generates “change events” corresponding to the locations where increased brightness has been detected (e.g., all the white pixels in the grid 520A). Each of the change events may include a timestamp. The Examiner interprets “predetermined striped light pattern” to be the patterns projected by the projection system since the claim is silent to the meaning of “predetermined.”).
Claims 17 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over Taamazyan et al. (U.S. Patent Pub. No. 20230084807, hereafter referred to as Taamazyan) in view of Mishra et al. (U.S. Patent No. 10,282,902, hereafter referred to as Mishra) in further view of Stoppa et al. (U.S. Patent Pub. No. 20180211373 hereafter referred to as Stoppa).
Regarding Claim 17, Taamazyan in view of Mishra discloses the method according to claim 14, further includes: obtaining the motion information of the depth data measurement device as the relative motion information (Col. 10, lines 50-57, Mishra teaches leveraging the relative movement of the object with respect to an imaging device (e.g., where the object is stationary and the imaging device is in motion).), and calculating the surface depth information of the imaging object based on the positions of pixels corresponding to the valid signals (Paragraph [0006], Taamazyan teaches computing a plurality of depths of surfaces imaged by the event camera at the event pixels associated with the first change events to generate a depth map. The Examiner interprets “event pixels” to be pixels corresponding to the valid signals (change in light exceeds threshold).), the timestamps of those valid signals (Paragraphs [0043], [0048], Taamazyan teaches an event camera is a type of image capture device that captures the change in brightness at each pixel instead of capturing the actual brightness value at each pixel. When a given pixel detects a change in the received light that exceeds a threshold value, the pixel generates an event, where the event is timestamped and may indicate the change in brightness at the pixel (e.g., brighter or darker) and in some cases, the magnitude of that change. Pixel-level events include the row and column of the pixel that generated the event (x, y) coordinates of the pixel within the image sensor, whether the pixel-level event was an ON event or an OFF event, and a timestamp of the pixel-level event.), as well as the relative motion information (Col. 9, lines 8-15 and lines 18-30, Mishra teaches using the known velocity (e.g., direction and speed) of the conveyor and the object in motion thereon, the 3D model (surface model) of the object defined using the first set of imaging data may be projected forward in a direction and by a distance of travel of the object defined by a product of a speed and an elapsed time between the first time and the second time, e.g., by advancing each of the vertices of the point cloud in the direction and by the distance. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets that 3D models (surface models) of the object(s) include “surface depth information” since they are a representation of depths to the object (see Col. 8, lines 38-44).) further includes: calculating the depth information of the surface of the object at the corresponding timestamp based on the pixel positions corresponding to the valid signals (Paragraphs [0006], [0043], Taamazyan teaches computing a plurality of depths of surfaces imaged by the event camera at the event pixels associated with the first change events to generate a depth map. Each pixel associated with a change event is timestamped.);
Taamazyan in view of Mishra does not explicitly disclose determining the status of the surface position of the object corresponding to each timestamp based on the depth information of the surface of the object at multiple timestamps and the motion information.
Stoppa is in the same field of art of capturing depth data of objects moving along a conveyor belt. Further, Stoppa teaches determining the status of the surface position of the object (Paragraph [0156], Stoppa teaches detecting defects in the construction of the surface of the object by inspecting the shape, size, and depth of creases. The Examiner interprets a defect located on the surface of the object to be a “status”. Further, the “surface position” is interpreted to be the location of the defect.) corresponding to each timestamp based on the depth information of the surface of the object at multiple timestamps (Paragraphs [0115-116], Stoppa teaches capturing several range images of a moving object at different times from one or more fixed cameras (such as depth cameras/array of depth cameras) placed at fixed locations. Multiple depth images can be taken of the object while the object is translating.) and the motion information (Paragraph [0121], Stoppa teaches precisely time-stamping the images, and estimating the object motion between time stamps of the images (e.g., if the timestamps are synchronized with the movement of the conveyor belt).).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Taamazyan in view of Mishra by identifying defects on the surface of the object as it moves along a conveyor belt that is taught by Stoppa, to make the invention that automates the detection of anomalous or defective objects using depth measurement data; thus, one of ordinary skilled in the art would be motivated to combine the references since automating defect detection in a factory or manufacturing environment for example, can improve the effectiveness of quality assurance and reduce the cost of staffing for quality assurance programs (Stoppa, Paragraph [0158]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 18, Taamazyan in view of Mishra in further view of Stoppa discloses the method according to claim 17, where determining the status of the surface position of the object (Paragraph [0156], Stoppa teaches detecting defects in the construction of the surface of the object by inspecting the shape, size, and depth of creases. The Examiner interprets a defect located on the surface of the object to be a “status”. Further, the “surface position” is interpreted to be the location of the defect.) corresponding to each timestamp based on the depth information of the surface of the object at multiple timestamps (Paragraphs [0115-116], Stoppa teaches capturing several range images of a moving object at different times from one or more fixed cameras (such as depth cameras/array of depth cameras) placed at fixed locations. Multiple depth images can be taken of the object while the object is translating.) further includes: comparing the depth information of the surface of the object at multiple timestamps with the depth information of the surface positions of the object (Paragraph [0024], Stoppa teaches detecting one or more defects by aligning the 3D multi-view model with a reference model and comparing the 3D multi-view model to the reference model to compute difference between corresponding regions of the 3D multi-view model and the reference model and detecting a defect when one or more of the differences exceeds a threshold.) corresponding to the predetermined depth distribution pattern to determine the surface status of the object (Paragraph [0030], Stoppa teaches comparing each of the features to a corresponding previously observed distribution of values of the feature and assigning a clean classification in response to determining that all of the values of the features are within a typical range or assigning a defect classification for each feature of the plurality of features that are in outlier portions of the corresponding previously observed distribution.); and/or synthesizing the actual surface depth distribution of the object based on the depth information of the surface of the object at multiple timestamps and the surface positions of the object corresponding to timestamps, and determining the surface status based on the surface depth distribution of the object (Under BRI, the Examiner interprets “and/or” to mean only one of the two limitations is required (comparing and/or synthesizing limitation.).
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Warashina (U.S. Patent Pub. No. 20230296373) teaches a 3D measurement device comprising a projection unit that projects a light onto an object which performing scanning with the light, a reception unit that receives the reference light reflected on the object, and a time setting unit that sets a scanning time extent of the reference light in accordance with a distance-measuring range for each prescribed section of the light reception unit. A 3D information calculation unit calculates 3D information about the object by means of triangulation.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYDNEY L BLACKSTEN whose telephone number is (571)272-7120. The examiner can normally be reached 8:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYDNEY L BLACKSTEN/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights