DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 is objected to because of the following informalities: in line 11, “the outline of the measured and reference objects” should be changed to read “the outlines of the object to be measured and the reference object”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 is rejected for a number of reasons which are given below:
Claim 1 recites “capturing a set of images…from the front or back, or from above, and a side view.” Does this require that the set of images must contain at least two images, one containing a side view of the object to be measured and another containing either a front, back, or above view of the object to be measured? Is another meaning intended? This will be the interpretation used for examination purposes.
Claim 1 recites that the step of “capturing a set of images” occurs “while placing a reference object…in the frame”. As written, it seems to require that the act of capturing a set of images and the act of placing a reference object in the image frame must occur simultaneously. Is this Applicant’s intent? The examiner assumes that it is not; rather, for examination purposes the examiner will assume that the claim requires only that each image must contain a reference object with known dimensions and of a known shape in the frame.
Claim 1 recites that the reference object in the frame must be placed such that “the dimensions are readable.” What does it mean for the dimensions to be “readable” in this context? In what situation would the dimensions be unreadable? Does Applicant intend the term “readable” to be synonymous with “determinable”? This will be the interpretation used for examination purposes.
Line 4 of claim 1 recites “the image”. This is confusing because a “set of images” was referred to earlier in the claim, therefore it is unclear which image is being referred to. For examination purposes it will be assumed that “the image” should be replaced with “each image”.
Claim 1 recites an “object to be measured,” an “object being measured,” and a “measured object”. It is unclear whether these refer to the same object or to different objects; for examination purposes, the examiner will assume that each reference refers to the same object. If this is Applicant’s intent, the examiner recommends using consistent language throughout to avoid ambiguity.
Claim 1 recites “calculating the actual dimension of the object [to be measured]”. In this quote, “the actual dimension” does not have antecedent basis. For examination purposes, it will be assumed that the above quote should be replaced with “calculating one or more physical dimensions of the object to be measured”.
Claim 1 recites “determining physical dimensions of the [object to be measured]”. Does Applicant intend for this to refer to the same dimensions as the “one or more physical dimensions” calculated as recited above? This will be the interpretation used for examination purposes.
Claim 1 recites “the perspective distortions” which has no antecedent basis. For examination purposes, it will be assumed that “the perspective distortions” should be replaced with “perspective distortions”.
Claim 1 recites that the step of determining the one or more dimensions of the object to be measured is based on “the determined apparent dimensions of the object [to be] measured”, which has no antecedent basis. The examiner believes that this refers to the fact that the relative dimensions of the object to be measured may be distorted in an image due to the angle at which the image was taken. However, it seems that these limitations are already met by preceding claim language; if an object is imaged, its apparent dimensions are determined, and the determination of the one or more dimensions of the object is based on the images taken. Does Applicant intend to recite that the determination of the one or more dimensions of the object to be measured is based on the detected outline of the object to be measured? This will be the interpretation used for examination purposes.
Considering the above issues, the examiner believes it would improve transparency to rewrite the claim language to reflect the interpretations which will be used for examination purposes. The examiner will use the following rewrite of claim 1 for examination purposes:
A method of measuring the linear dimensions of an object, comprising:
capturing a set of images of an object to be measured from the front, back, or above, and from a side, each image containing a reference object with known dimensions and of a known shape in the frame such that the reference object’s dimensions are discernible;
processing the set of images with a computational algorithm; and
calculating one or more physical dimensions of the object to be measured, wherein
the set of images are electronically transferred to a server to the computational algorithm;
each image is sequentially processed with computer vision algorithms and an artificial neural network, said processing including:
detecting a pixel mask and outline of the reference object;
approximating the outline of the reference object with a polygon;
detecting a pixel mask and outline of the object being measured;
detecting extreme points of the outlines of the object to be measured and the reference object; and
using a projective transformation, determining the one or more physical dimensions of the object to be measured taking into account perspective distortions and based on the detected outline of the object being measured.
Examiner’s Statement Regarding Subject Matter Eligibility Under 35 USC § 101
Claim 1 is eligible under 35 U.S.C. 101.
At Step 1 of the 101 analysis, claim 1 is directed to one of the statutory categories of invention.
At Step 2A, Prong One, the judicial exceptions are bolded in the copy of claim 1 below:
A method of measuring the linear dimensions of an object, comprising:
capturing a set of images of an object to be measured from the front, back, or above, and from a side, each image containing a reference object with known dimensions and of a known shape in the frame such that the reference object’s dimensions are discernible;
processing the set of images with a computational algorithm; and
calculating one or more physical dimensions of the object to be measured, wherein
the set of images are electronically transferred to a server to the computational algorithm;
each image is sequentially processed with computer vision algorithms and an artificial neural network, said processing including:
detecting a pixel mask and outline of the reference object;
approximating the outline of the reference object with a polygon;
detecting a pixel mask and outline of the object being measured;
detecting extreme points of the outlines of the object to be measured and the reference object; and
using a projective transformation, determining the one or more physical dimensions of the object to be measured taking into account perspective distortions and based on the detected outline of the object being measured.
Calculating dimensions of an object refers to a mathematical process; detecting an outline of an object in an image, approximating it with a polygon, and determining extreme points of the outline are all encompassed by mental or mathematical operations depending on the complexity of the operations. Determining a dimension of an object in an image accounting for perspective distortions and based on a detected outline, encompasses a mathematical process that can be performed by a person with pen and paper. Finally, using a projective transformation encompasses a mathematical operation.
At Step 2A, Prong Two, the additional elements involve capturing images of an object with a reference object in the frame; processing the images with an algorithm, on a server with computer vision and an artificial neural network (ANN), and detecting pixel masks of objects in an image. These additional elements apply computers and computer algorithms to images to calculate one or more dimensions of an object, which is considered a practical, useful result. Furthermore, the additional elements require physically capturing images of the object to be measured with a reference object in the frame. The examiner considers that the additional elements integrate the judicial exceptions into the practical application of determining an object’s size from a 2D image of the object.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Garcia (US 20140300722 A1) in view of Qin ("U2-Net: Going deeper with nested U-structure for salient object detection").
Regarding claim 1, Garcia discloses a method of measuring the linear dimensions of an object (¶4: "The invention relates to the field of measurement, specifically, the use of novel image-based measurement tools to accurately determine the dimensions of objects that have been imaged"; ¶10: "The methods of the invention can be utilized to determine the size of an object in one dimension, i.e. a linear measurement. Typical linear measurements are length, width, and height.”), comprising:
capturing a set of images of an object to be measured (¶11: " In some embodiments, multiple measurements of the item from different angles can be analyzed in order to determine the desired parameter." ¶71: "the image analysis means can be utilized to integrate measurements from multiple images…a single two-dimensional image cannot capture those aspects of the item which are outside the plane of the photograph. In such cases, it may be advantageous to take multiple images of the item from different vantage points so that multiple attributes of a three-dimensional item can be measured."), each image containing a reference object with known dimensions and of a known shape in the frame such that the reference object’s dimensions are discernible (¶13: "a reference standard, i.e., an object of known dimensions, is placed on, near, or otherwise in plane with the item to be measured. Secondly, an image capture means, such as a digital camera, is used to capture an image of the item and the reference standard in the same image.");
processing the set of images with a computational algorithm (¶13: "Third, a processing means, guided by user delineations, analyzes the image" or images); and
calculating one or more physical dimensions of the object to be measured (¶13: "Lastly, using the previously calculated relationship between image size and actual size, and the image size of the item of interest, the actual size of the item of interest is determined."), wherein
the set of images are electronically transferred to the computational algorithm (Abstract: a computer program is used to enable measurement of the objects);
each image is sequentially processed with computer vision algorithms (¶64: "Object recognition software may also be utilized to automatically delineate the endpoints or borders of the target object. Object recognition (sometimes called machine vision) software can delineate objects in an image using color and pattern recognition algorithms, as well as other tools." Machine vision is synonymous with computer vision.), said processing including:
detecting an outline of the object being measured (from ¶64, the borders of the objects in the image can be detected); and
using a projective transformation (¶69: "a matrix of correction factors which correspond to the degree and directionality of distortion for a particular camera lens can be stored in the image analysis means and then applied to the measurement calculations after mapping the location of the measured item to account for the distortion expected in that portion of the image." The matrix of correction factors is a kind of projective transformation), determining the one or more physical dimensions of the object to be measured (see ¶4 referenced above) taking into account perspective distortions (¶69: "In calculating the size of the object, the image analysis means can utilize various correction factors to account for distortions of the image which might otherwise introduce errors into the measurement") and based on the detected outline of the object being measured (¶67: "Once the image sizes of the target object and the reference standard object are known, and the actual dimensions of the reference standard object have been retrieved or inputted, the actual size of the target object may be calculated." From context it is clear that the determination is based on the outline detected in previous paragraphs.).
Garcia does not explicitly disclose that the set of images include images from the front, back, or above, and from a side. However, it would have been obvious to take such measurements since all such image angles are common and it would be necessary to take at least two images to unambiguously capture all three dimensions of a three-dimensional object in the image plane.
Garcia does not explicitly state that the computational algorithm is on a server; however, since servers are common in computing and can perform a variety of general tasks on a network, it would have been obvious to cause the computational algorithm to be on a server.
Garcia does not explicitly disclose that the processing with computer vision algorithms includes detecting an outline of the reference object; however ¶64 of Garcia explicitly discloses using object recognition software to delineate borders of a target object, and it would have been obvious to use the software for a reference object as well to automatically find the reference object in the image (note that other paragraphs, such as ¶60, describe delineating borders of the reference standard). Additionally, it would have been obvious to approximate the outline of the reference object with a polygon if the reference object is polygonal (¶149 gives an example in which the reference object is an L-shaped bend, which is a polygon).
Garcia does not explicitly recite detecting extreme points of the outlines of the object to be measured and the reference object. However, Garcia does teach that extreme points can be delineated (see ¶64 referenced above), and it would have been obvious to detect the extreme points of outlines as part of determining a linear dimension of interest (for example, ¶10 describes how "the height of a person is commonly understood to refer to the one-dimensional distance from the bottom of the person's feet to the top of their head." It would be useful both to identify a person in an image by detecting their outline, and to detect extreme points of the outline to determine the person's height).
Garcia does not explicitly disclose that each image is processed with an ANN, nor that the processing includes detecting pixel masks of the reference object and the object to be measured.
Qin teaches that ANNs have been used to significantly improve salient object detection in an image (Pg. 1, column 1, first paragraph), and that a number of object detection methods detect pixel masks of objects (Pg. 10, Fig. 7 depicts a number of pixel masks of objects detected by a number of different methods).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate the teachings of Qin with the invention of Garcia by processing each image with an ANN and causing the processing to include detecting pixel masks of the reference object and the object to be measured. ANNs are useful for object detection from images, and detecting a pixel mask of an object is a common method of object detection software.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ETHAN WESLEY EDWARDS whose telephone number is (571)272-0266. The examiner can normally be reached Monday - Friday, 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Schechter can be reached at (571) 272-2302. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ETHAN WESLEY EDWARDS
Examiner
Art Unit 2857
/E.W.E./ Examiner, Art Unit 2857
/ANDREW SCHECHTER/ Supervisory Patent Examiner, Art Unit 2857
Prosecution Insights