DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/24/2026 has been entered.
Response to Arguments
Applicant’s arguments, filed 06/24/2026, with respect to the claims 1-5 and 7-21 have been fully considered but are moot because the arguments do not apply to the current references/combination of references being used in the current rejection. The objection(s) for claim 1 for minor informalities has been withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5, 8-11, 16-17, and 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over AARTS et al. (US 20120183204 A1), hereinafter referenced as AARTS in view of FATHI (US 20160364885 A1), hereinafter referenced as FATHI and in further view of WEXLER et al. (US 20170284799 A1), hereinafter referenced as WEXLER.
Regarding claim 1, AARTS explicitly teaches a method (Fig. 3. Paragraph [0026]-AARTS discloses FIG. 3 illustrates a process performed by the image conversion system 100. Please also see Fig. 4-7) implemented with instructions executed by a processor (Fig. 2, #202 called a central processing unit. Paragraph [0022]. In paragraph [0020]-AARTS discloses FIG. 1 depicts a block diagram of an image conversion system 100 suitable for use with the methods and systems consistent with the present invention. The image conversion system 100 comprises a plurality of computers 102, 104, and 106 connected via a network 108. In paragraph [0022]-AARTS discloses computer 102 comprises a central processing unit (CPU) 202, a secondary storage device 208, and a memory 210), comprising:
processing a single digital image (Fig. 4, #400 called an image. Paragraph [0034]. In paragraph [0026]-AARTS discloses in step 302, an image is captured by an image capturing unit communicatively coupled to a computer 102, 104, or 106. The image is transmitted to the image receiving unit 110 operating in the memory of the computer 102 in step 304 using any conventional information transferring method. In paragraph [0027]-AARTS discloses in step 306, information concerning the captured image is gathered by the information gathering unit 112. In step 308, the image analysis unit 114 determines the physical dimensions of the room in which the image was captured, based on the image information. Please also see Fig. 3) of an interior space (Fig. 4. Paragraph [0034]-AARTS discloses FIG. 4 depicts a two dimensional image 400 of a room captured by an image capture device. Please also see Fig. 4-5 and read paragraph [0036]);
determining dimensions of the detected object (Fig. 3. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. In paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts each object in the room into a three dimensional object by relating the dimensions of each object to the dimensions of the room and the position of each object within the room. Please also see Fig. 4-5 and read paragraph [0055]);
applying image segmentation to the single digital image to produce a segmented image (Fig. 3. Paragraph [0028]-AARTS discloses the image analysis unit 114 may identify objects in an image by analyzing the pixels in the image to determine lines where the pixel colors change from one color to another. Please also see Fig. 4-5 and read paragraph [0036-0038]);
detecting edges in the segmented image to produce a combined output image produced from said single digital image (Fig. 5. Paragraph [0036]-AARTS discloses the image analysis unit 114 may also use a line analysis algorithm to identify where the lines that forms the intersections between the walls 402, 404, and 406 in the image. The line analysis algorithm may include a Hough transform algorithm or any other image line analysis algorithm that is known in the art. Please also see Fig. 3-4 and read paragraph [0028-0030 and 0047]);
Although AARTS explicitly teaches identifying at least one detected object (Fig. 3. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. Please also read paragraph [0055]) within the single digital image (Fig. 3. Paragraph [0027]-AARTS discloses in step 302, an image is captured by an image capturing unit communicatively coupled to a computer 102, 104, or 106. The image may be captured using any conventional image capturing device such as, but not limited to, a digital camera or any other device capable of capturing an image and converting the image into a digital format. Please also read paragraph [0055-]), wherein said at least one detected object comprises a standard construction object having standard known dimensions (Fig. 3. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. Please also read paragraph [0055]) within the single digital image (Fig. 3. Paragraph [0055]-AARTS discloses FIG. 7 illustrates a process of inserting a new object into the converted image. In step 702, a plurality of objects is retrieved from the object storage unit 214 along with sizing information for the object such as height, width and depth).
AARTS fails to explicitly teach wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate.
However, FATHI explicitly teaches wherein said at least one detected object comprises a standard construction object having standard known dimensions (Fig. 1B. Paragraph [0067]-FATHI discloses the present invention provides accurate 3D digital representations of at least one object in a scene. In paragraph [0082]-FATHI discloses measurements of the at least one object of interest can be obtained without use of, or in addition to, a marker. In this regard, the invention utilizes an internal or "intrinsic" reference. In paragraph [0083]-FATHI discloses with regard to an intrinsic reference derived from the focal length of the image-capture device most existing image-capture devices (e.g., cameras) comprise a short depth of field, resulting in images which appear focused only on a small 3D slice of the scene. In paragraph [0084]-FATHI discloses in a further aspect of the intrinsic reference feature of the present invention, a library of standard object identities and sizes can be included in the software associated with the image-capture device to provide data from which measurement data for the at least one object of interest can be derived. Please also see Fig. 3 and read paragraph [0081-0082 and 0085]), wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate (Fig. 3. Paragraph [0084]-FATHI discloses for example, if a single toggle light switchplate, which has a standard US size of 4.5 inches (11.42 cm) in height and 2.75 inches (6.985 cm) in width, appears in a scene with an object of interest, the known standard dimensions of this switchplate can be used as an intrinsic reference to provide a point of reference from which the dimensions of the object of interest can be derived. In paragraph [0085]-FATHI discloses if the identified reference object that will serve as the intrinsic reference for providing measurement of an object of interest present in the scene is a switchplate cover, the system 101 will elicit and receive the specification of an object to be used for dimensional calibration, and the user will select the switchplate cover to serve as the intrinsic reference. Therefore, it would have been obvious to one of ordinary skill in the art to use a group of standard construction objects consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate. The types of electrical wall-plates listed represent the most common and well-known categories of wall plates. Moreover, electrical wall plates are ubiquitous construction objects that are often in close proximity to other more important target objects. They also offer highly consistent and/or standardized dimensions. Thus, it would be obvious to use this specific group of electrical wall plates given they are strong potential candidates for a reference object. This would be an obvious extension of the functionality of FATHI and improve the accuracy and consistency of object detection, identification and measurement);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image; determining dimensions of the detected object; applying image segmentation to the single digital image to produce a segmented image, with the teachings of FATHI of having wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate.
Wherein having AARTS method having identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate; wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and FATHI concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while FATHI provides systems and methods for an object-oriented building-development tool that utilizes modeling concepts, information technology and software interoperability to improve photogrammetry and reduce the amount of data needed to produce more accurate 3D digital representations. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and FATHI et al. (US 20160364885 A1), Abstract, Paragraph [0010, 0030 and 0047].
Although AARTS explicitly teaches applying dimensions to the transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan (Fig. 3. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room. The image conversion unit 116 generates a three dimensional plane for each wall of the room, and stores these planes in the memory 210. In addition, the image conversion unit 116 converts each object in the room into a three dimensional object by relating the dimensions of each object to the dimensions of the room and the position of each object within the room. In paragraph [0030]-AARTS discloses in step 318, the image conversion unit 116 presents the converted image to the user via the GUI 232 coupled to the user device 104 or 106. Please also read paragraph [0038-0041 and 0055]).
AARTS in view of FATHI fails to explicitly teach applying geometric transformation, field of view and depth correction to the combined output image to correct for image distortion to produce a geometrically transformed digital image, wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object; and applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan.
However, WEXLER explicitly teaches applying geometric transformation, field of view and depth correction (Fig. 13A-B. Paragraph [0030]-WEXLER discloses FIGS. 13A and 13B are flow diagrams illustrating a method for determining a perspective transform. Further in paragraph [0234]-WEXLER discloses depth offset correction may be used as described supra for measuring target object dimensions that are slightly offset from the plane to which the reference object is adhered. Such depth offset corrections may be obtained using the methods described supra, incorporating the difference in pixel dimensions found for pane ancillary objects and adhering objects which are the same physical size and that reside in the two substantially parallel planes. Please also read paragraph [0153-0163 and 0244-0246]) to the combined output image (Fig. 26. Paragraph [0078]-WEXLER discloses a block diagram illustrating an example room in which an end user obtains a digital image of sample window is shown in FIG. 3. The distance measurement mechanism enables the automatic measurement of dimensions for a window part from a digital image of the window that includes a reference object in substantially the same plane as the constraint dimension associated with the window. The end user 131 takes a photograph, using a digital image acquisition device such as digital camera 133, which includes a window 130 and the reference object 132. Knowledge of the reference object dimensions is used to calculate any dimensions needed for fabrication of one or more parts for fenestration (wherein reference objects and target objects may be standard objects of known dimensions and include walls, furniture, pictures, holes/openings in walls, etc.). In paragraph [0118]-WEXLER discloses in some cases it may be desirable to capture only a single image. In paragraph [0244]-WEXLER discloses once the reference object is located, the adhering objects are found in the digital image by color and/or shape segmentation of the image (step 541). In paragraph [0245]-WEXLER discloses an edge detector, such as the well-known Canny edge detector and probabilistic Hough transform may be used to create a consensus line within each region of interest (step 552). Please also read paragraph [0153-0157 and 0238]) to correct for image distortion to produce a geometrically transformed digital image (Fig. 26. Paragraph [0245]-WEXLER discloses once the preliminary perspective transform is obtained, it is adjusted so that the center point and center geometric mean scale of the original image is preserved such that the aspect ratio of the reference object is maintained. This is achieved by balancing the relative scaling of the 2D transform (step 560), normalizing the perspectivity matrix (step 562) and preserving the image center location (step 564). Please also read paragraph [0153-0163]), wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object (Fig. 2. Paragraph [0157]-WEXLER discloses after the transform is determined, repeated correction of the perspective distortion in the triage resolution image is optional. In paragraph [0167]-WEXLER discloses fiducial size differences relative to the reference object (step 420) may be used to make adjustments needed to account for out-of-plane scale distortions induced by the use of the pane plane projection transform and may be used to calculate corrected pixels per inch for the plane of interest to allow for more accurate measurement of such dimensions (wherein perspective transformation is a form of projective transformation). Please also read paragraph [0155, 0169, 0172, 0180]); and
applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object (Fig. 26. Paragraph [0246]-WEXLER discloses a flow diagram illustrating an example method for calculating target object dimensions is shown in FIG. 27. The self-adhering pane ancillary objects are re-found in the corrected/transformed image (step 570). The pixel dimensions of the transformed bounding box 465 created from the pane ancillary objects are then determined (step 574). The pixel dimensions are then converted to the pane physical dimensions using the reference object pixel scale factor (i.e. pixels per unit length) (step 576). In paragraph [0247]-WEXLER discloses the method for finding ancillary objects illustrated in FIG. 25 may also be used to find ancillary objects used to define non-transparent target objects such as walls or portions thereof. In such cases, an ancillary object edge adhered to the wall may be used to define a wall corner, the edge defined where the wall and floor or the wall and ceiling meet. Please also read paragraph [0154-0160, 0177-0180]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate; determining dimensions of the detected object; applying image segmentation to the single digital image to produce a segmented image, with the teachings of WEXLER of having applying geometric transformation, field of view and depth correction to the combined output image to correct for image distortion to produce a geometrically transformed digital image, wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object; and applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan.
Wherein having AARTS method applying geometric transformation, field of view and depth correction to the combined output image to correct for image distortion to produce a geometrically transformed digital image, wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object; and applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and WEXLER concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while WEXLER provides systems and methods that processes one or more digital images to provide improved measurement capabilities. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and WEXLER et al. (US 20170284799 A1), Abstract, Paragraph [0045].
Regarding claim 2, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS further teaches wherein the dimensionalized floorplan is a three-dimensional construction image (Fig. 3. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room. The image conversion unit 116 generates a three dimensional plane for each wall of the room, and stores these planes in the memory 210. In addition, the image conversion unit 116 converts each object in the room into a three dimensional object by relating the dimensions of each object to the dimensions of the room and the position of each object within the room. Further in paragraph [0042]-AARTS discloses in step 514, a virtual three-dimensional representation of the room in the image 400 is stored in memory 210 of the computer 102 along the optimal vector x, which defines the dimensions of the room, and the information previously gathered in step 508 that defines the appearance of the room).
Regarding claim 5, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS further teaches wherein determining dimensions of the detected object is based upon a reference database (Fig. 6. Paragraph [0028] In step 310, the image analysis unit 114 identifies objects in the image. The image analysis unit 114 may identify objects in an image by analyzing the pixels in the image to determine lines where the pixel colors change from one color to another. The image analysis unit 114 may also identify objects by comparing areas identified in the image to a database of known images. Further in paragraph [0032]-AARTS discloses in step 322, the GUI 232 on the user device 104 or 106 displays a list of objects to insert into the converted image from the object storage unit 210 in the secondary storage 208 of the computer 102. The objects in the object storage unit 210 include information concerning each object listed including, but not limited to, the dimensions of the object, the color of the surfaces of the object, the composition of each surface and the reflective characteristics of each surface).
Regarding claim 8, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS further teaches wherein the geometrically transformed digital image (Fig. 3. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room. In paragraph [0036]-AARTS discloses FIG. 5 illustrates a process of determining the dimensions of a room from an image 400. In step 504, the information gathering unit 114 receives basic dimensional information of the image via the GUI 232, and from the information stored in the image such as EXIF information. The image analysis unit 114 may also use a line analysis algorithm to identify where the lines that forms the intersections between the walls 402, 404, and 406 in the image (wherein the line analysis algorithm may include a Hough transform algorithm). Please also see Fig. 4-5) is a pixel image (Fig. 5. Paragraph [0037]-AARTS in step 506, the image analysis unit 114 identifies the walls 402, 404, and 406 displayed in the image 400 based on the information gathered from the information gathering unit 112. The walls 402, 404, and 406 are identified in the image as the pixels in the captured image contained in the non-self-intersecting polygons formed by pairs of neighboring lines which form the intersection between walls. In paragraph [0038]-AARTS discloses in step 508, the image analysis unit 114 gathers information on each wall 402, 404, and 406. To gather this information, the image analysis unit 114 systematically analyzes the pixels in each wall to determine the colors of each wall, and the relative location of each color on each wall).
Regarding claim 9, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS further teaches further comprising: positioning a bounding box around the detected object (Fig. 6A, # 602 called a removal area. Paragraph [0045]); and using the bounding box (Fig. 6A, # 602 called a removal area. Paragraph [0045]) and the combined output image (Fig. 6A, #400 called an image. Paragraph [0045]. Please also read paragraph [0036, 0038 and 0041]) to determine a digital perimeter of the detected object (Fig. 6A. Paragraph [0047]-Aarts discloses in step 612, the image analysis unit 114 identifies objects 604 within the removal area 602. The image analysis unit 114 may use any known object identification technique such as edge detection, image matching, or any other known image identification technique).
Regarding claim 10, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 9, AARTS further teaches further comprising using a geometrical correction technique in determining an adjusted digital perimeter of the detected object (Fig. 6A. Paragraph [0046]-Aarts discloses in step 610, the image analysis unit 114 identifies at least one sample area 604 for the identified removal area 602. The sample area 606 may be identified using the same techniques as identifying the removal area 602. Further in paragraph [0048]-Aarts discloses in step 614, the image analysis unit 114 divides the removal area 602 into target patches. The target patches may be of the same size and shape. Each of the target patches represents a portion of the removal area 602 where the pixel information in that area is removed and replaced by the pixel information from the sample area 606. Additionally, in paragraph [0051]-Aarts discloses single linear gradient of random size, and of random orientation, is applied to each potential sample patch. Subsequently, each potential sample patch is multiplied with an intensity correction factor. Please also read paragraph [0047]).
Regarding claim 11, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 10, AARTS further teaches wherein the geometrical correction techniques compare the determined digital perimeter of the detected object to dimensions or geometric properties of the detected object to calculate an angular offset therebetween to determine the adjusted digital perimeter (Fig. 6A. Paragraph [0047]-AARTS discloses in step 612, the image analysis unit 114 identifies objects 604 within the removal area 602. The image analysis unit 114 may use any known object identification technique such as edge detection, image matching, or any other known image identification technique. Further in paragraph [0048]-AARTS discloses in step 614, the image analysis unit 114 divides the removal area 602 into target patches. Additionally in paragraph [0050]-AARTS discloses in step 618, a group of sample patches is generated from the identified sample area 606. Each sample patch in the group may be a rectangle of a fixed size that is twenty five percent larger than the size of each target patch. The group of sample patches is created by visiting random locations in each sample area 606, and extracting pixel information from each sample patch. Please also read paragraph [0051]).
Regarding claim 16 AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS further teaches wherein the single digital image comprises a pixel image (Fig. 4. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. The image analysis unit 114 may identify objects in an image by analyzing the pixels in the image to determine lines where the pixel colors change from one color to another. The image analysis unit 114 may also identify objects by comparing areas identified in the image to a database of known images. In paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room).
Regarding claim 17, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 16, AARTS further teaches wherein the pixel image comprises a single two-dimensional (2D) pixel image (Fig. 4. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room).
Regarding claim 19, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS in view of FATHI fail to explicitly teach wherein said projective transformation is planar projective transformation.
However, WEXLER explicitly teaches wherein said projective transformation is planar projective transformation (Fig. 26. Paragraph [0121]-WEXLER discloses a measureable image is an image in which rectangular objects in the world appear rectangular in an image; for example, an image of a rectangular object directly in front of a camera where the axis of the lens is perpendicular to the rectangular object. A projective transformation is a mapping from points in one coordinate system to points in another coordinate system which preserves collinearity; for example, a pinhole camera photograph of a planar object on the image plane of the camera. A projectivity matrix is a matrix which is used to encapsulate the calculations needed to perform a projective transformation; for example a 3×3 matrix that is used to convert locations (u, v) in one plane to locations (x, y) in another plane by adding a third coordinate of 1 to a source point (u,v,l) and computing three coordinates (hx, hy, h) by matrix-vector multiplication from which the coordinates of the destination point (x, y) are computed by division. In paragraph [0154]-WEXLER discloses flow diagrams illustrating a method for determining a perspective transform are shown in FIGS. 13A and 13B (wherein perspective transformation is a form of projective transformation, and the perspective transformation is planar projective transformation). Please also read paragraph [0118, 0153-0157 and 0245-0246]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of WEXLER of having wherein said projective transformation is planar projective transformation.
Wherein having AARTS method having wherein said projective transformation is planar projective transformation.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and WEXLER concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while WEXLER provides systems and methods that processes one or more digital images to provide improved measurement capabilities. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and WEXLER et al. (US 20170284799 A1), Abstract, Paragraph [0045].
Regarding claim 20, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS in view of FATHI fail to explicitly teach wherein applying said projective transformation to the single digital image provides an angle of view, height, and tilt of an image capturing device used to capture said single digital image, wherein said geometrically transformed digital image is produced based, at least in part, on said angle of view, height, and tilt.
However, WEXLER explicitly teaches wherein applying said projective transformation to the single digital image (Fig. 26. Paragraph [0118]-WEXLER discloses in some cases it may be desirable to capture only a single image. In paragraph [0245]-WEXLER discloses once the preliminary perspective transform is obtained, it is adjusted so that the center point and center geometric mean scale of the original image is preserved such that the aspect ratio of the reference object is maintained. This is achieved by balancing the relative scaling of the 2D transform (step 560), normalizing the perspectivity matrix (step 562) and preserving the image center location (step 564). Please also read paragraph [0118, 0153-0163 and 0246]) provides an angle of view, height, and tilt of an image capturing device used to capture said single digital image, wherein said geometrically transformed digital image is produced based, at least in part, on said angle of view, height, and tilt (Fig. 26. Paragraph [0182]-WEXLER discloses using the reference object and ancillary objects, in offset planes, 3D projective geometric modeling together with information regarding the relationship between camera lens focal length and sensor size, typically described using a 35 mm equivalent focal length, may be used to analyze the 2D images obtained in the present invention to provide measurement estimates between and within the offset plane. Although it is technically possible to determine a pose and position of the reference object or pane relative to the camera by analysis of the projectivity matrix, this commonly performed geometric scene reconstruction task may be accomplished directly using routines such as solvePnP provided in OpenCV given the same source and destination data together with camera intrinsic parameter data. The routine provides a rotation vector r and a translation vector T that fully describe the relationship between the set of world and camera coordinate systems. Further in paragraph [0185]-WEXLER discloses when doing projective geometry, a fourth row is added to account for ambiguity of locations of points in the real world along lines through the camera coordinate system origin, resulting in a model using homogeneous coordinates wherein points [x y z l], [hx hy hz h]. The model equation relating coordinates [u v w] and [x y z] is now nonlinear, but can be expressed using a linear set of equations encapsulated into a 3D projectivity matrix).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of WEXLER of having wherein applying said projective transformation to the single digital image provides an angle of view, height, and tilt of an image capturing device used to capture said single digital image, wherein said geometrically transformed digital image is produced based, at least in part, on said angle of view, height, and tilt.
Wherein having AARTS method having wherein applying said projective transformation to the single digital image provides an angle of view, height, and tilt of an image capturing device used to capture said single digital image, wherein said geometrically transformed digital image is produced based, at least in part, on said angle of view, height, and tilt.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and WEXLER concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while WEXLER provides systems and methods that processes one or more digital images to provide improved measurement capabilities. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and WEXLER et al. (US 20170284799 A1), Abstract, Paragraph [0045].
Regarding claim 21, AARTS explicitly teaches a method (Fig. 3. Paragraph [0026]-AARTS discloses FIG. 3 illustrates a process performed by the image conversion system 100. Please also see Fig. 4-7) implemented with instructions executed by a processor (Fig. 2, #202 called a central processing unit. Paragraph [0022]. In paragraph [0020]-AARTS discloses FIG. 1 depicts a block diagram of an image conversion system 100 suitable for use with the methods and systems consistent with the present invention. The image conversion system 100 comprises a plurality of computers 102, 104, and 106 connected via a network 108. In paragraph [0022]-AARTS discloses computer 102 comprises a central processing unit (CPU) 202, a secondary storage device 208, and a memory 210), comprising:
processing a single digital image (Fig. 4, #400 called an image. Paragraph [0034]. In paragraph [0026]-AARTS discloses in step 302, an image is captured by an image capturing unit communicatively coupled to a computer 102, 104, or 106. The image is transmitted to the image receiving unit 110 operating in the memory of the computer 102 in step 304 using any conventional information transferring method. In paragraph [0027]-AARTS discloses in step 306, information concerning the captured image is gathered by the information gathering unit 112. In step 308, the image analysis unit 114 determines the physical dimensions of the room in which the image was captured, based on the image information. Please also see Fig. 3) of an interior space (Fig. 4. Paragraph [0034]-AARTS discloses FIG. 4 depicts a two dimensional image 400 of a room captured by an image capture device. Please also see Fig. 4-5 and read paragraph [0036]);
determining dimensions of the detected object (Fig. 3. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. In paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts each object in the room into a three dimensional object by relating the dimensions of each object to the dimensions of the room and the position of each object within the room. Please also see Fig. 4-5 and read paragraph [0055]);
applying image segmentation to the single digital image to produce a segmented image (Fig. 3. Paragraph [0028]-AARTS discloses the image analysis unit 114 may identify objects in an image by analyzing the pixels in the image to determine lines where the pixel colors change from one color to another. Please also see Fig. 4-5 and read paragraph [0036-0038]);
detecting edges in the segmented image to produce a combined output image produced from said single digital image (Fig. 5. Paragraph [0036]-AARTS discloses the image analysis unit 114 may also use a line analysis algorithm to identify where the lines that forms the intersections between the walls 402, 404, and 406 in the image. The line analysis algorithm may include a Hough transform algorithm or any other image line analysis algorithm that is known in the art. Please also see Fig. 3-4 and read paragraph [0028-0030 and 0047]);
Although AARTS explicitly teaches identifying at least one detected object (Fig. 3. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. Please also read paragraph [0055]) within the single digital image (Fig. 3. Paragraph [0027]-AARTS discloses in step 302, an image is captured by an image capturing unit communicatively coupled to a computer 102, 104, or 106. The image may be captured using any conventional image capturing device such as, but not limited to, a digital camera or any other device capable of capturing an image and converting the image into a digital format. Please also read paragraph [0055-]), wherein said at least one detected object comprises a standard construction object having standard known dimensions (Fig. 3. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. Please also read paragraph [0055]) within the single digital image (Fig. 3. Paragraph [0055]-AARTS discloses FIG. 7 illustrates a process of inserting a new object into the converted image. In step 702, a plurality of objects is retrieved from the object storage unit 214 along with sizing information for the object such as height, width and depth).
AARTS fails to explicitly teach wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate.
However, FATHI explicitly teaches wherein said at least one detected object comprises a standard construction object having standard known dimensions (Fig. 1B. Paragraph [0067]-FATHI discloses the present invention provides accurate 3D digital representations of at least one object in a scene. In paragraph [0082]-FATHI discloses measurements of the at least one object of interest can be obtained without use of, or in addition to, a marker. In this regard, the invention utilizes an internal or "intrinsic" reference. In paragraph [0083]-FATHI discloses with regard to an intrinsic reference derived from the focal length of the image-capture device most existing image-capture devices (e.g., cameras) comprise a short depth of field, resulting in images which appear focused only on a small 3D slice of the scene. In paragraph [0084]-FATHI discloses in a further aspect of the intrinsic reference feature of the present invention, a library of standard object identities and sizes can be included in the software associated with the image-capture device to provide data from which measurement data for the at least one object of interest can be derived. Please also see Fig. 3 and read paragraph [0081-0082 and 0085]), wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate (Fig. 3. Paragraph [0084]-FATHI discloses for example, if a single toggle light switchplate, which has a standard US size of 4.5 inches (11.42 cm) in height and 2.75 inches (6.985 cm) in width, appears in a scene with an object of interest, the known standard dimensions of this switchplate can be used as an intrinsic reference to provide a point of reference from which the dimensions of the object of interest can be derived. In paragraph [0085]-FATHI discloses if the identified reference object that will serve as the intrinsic reference for providing measurement of an object of interest present in the scene is a switchplate cover, the system 101 will elicit and receive the specification of an object to be used for dimensional calibration, and the user will select the switchplate cover to serve as the intrinsic reference. Therefore, it would have been obvious to one of ordinary skill in the art to use a group of standard construction objects consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate. The types of electrical wall-plates listed represent the most common and well-known categories of wall plates. Moreover, electrical wall plates are ubiquitous construction objects that are often in close proximity to other more important target objects. They also offer highly consistent and/or standardized dimensions. Thus, it would be obvious to use this specific group of electrical wall plates given they are strong potential candidates for a reference object. This would be an obvious extension of the functionality of FATHI and improve the accuracy and consistency of object detection, identification and measurement);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image; determining dimensions of the detected object; applying image segmentation to the single digital image to produce a segmented image, with the teachings of FATHI of having wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate.
Wherein having AARTS method having identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate; wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and FATHI concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while FATHI provides systems and methods for an object-oriented building-development tool that utilizes modeling concepts, information technology and software interoperability to improve photogrammetry and reduce the amount of data needed to produce more accurate 3D digital representations. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and FATHI et al. (US 20160364885 A1), Abstract, Paragraph [0010, 0030 and 0047].
Although AARTS explicitly teaches applying dimensions (Fig. 3. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room. Please also read paragraph [0042]) to the geometrically transformed digital image (Fig. 3. Paragraph [0036]-AARTS discloses the image analysis unit 114 may also use a line analysis algorithm to identify where the lines that forms the intersections between the walls 402, 404, and 406 in the image. The line analysis algorithm may include a Hough transform algorithm or any other image line analysis algorithm that is known in the art) at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan (Fig. 3. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room. The image conversion unit 116 generates a three dimensional plane for each wall of the room, and stores these planes in the memory 210. In addition, the image conversion unit 116 converts each object in the room into a three dimensional object by relating the dimensions of each object to the dimensions of the room and the position of each object within the room. In paragraph [0030]-AARTS discloses in step 318, the image conversion unit 116 presents the converted image to the user via the GUI 232 coupled to the user device 104 or 106. Please also read paragraph [0038-0041]).
Although AARTS explicitly teaches applying dimensions to the transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan (Fig. 3. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room. The image conversion unit 116 generates a three dimensional plane for each wall of the room, and stores these planes in the memory 210. In addition, the image conversion unit 116 converts each object in the room into a three dimensional object by relating the dimensions of each object to the dimensions of the room and the position of each object within the room. In paragraph [0030]-AARTS discloses in step 318, the image conversion unit 116 presents the converted image to the user via the GUI 232 coupled to the user device 104 or 106. Please also read paragraph [0038-0041]).
AARTS in view of FATHI fails to explicitly teach applying geometric transformation, field of view and depth correction to the combined output image to correct for image distortion to produce a geometrically transformed digital image, wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object; and applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan.
However, WEXLER explicitly teaches applying geometric transformation, field of view and depth correction (Fig. 13A-B. Paragraph [0030]-WEXLER discloses FIGS. 13A and 13B are flow diagrams illustrating a method for determining a perspective transform. Further in paragraph [0234]-WEXLER discloses depth offset correction may be used as described supra for measuring target object dimensions that are slightly offset from the plane to which the reference object is adhered. Such depth offset corrections may be obtained using the methods described supra, incorporating the difference in pixel dimensions found for pane ancillary objects and adhering objects which are the same physical size and that reside in the two substantially parallel planes. Please also read paragraph [0153-0163 and 0244-0246]) to the combined output image (Fig. 26. Paragraph [0078]-WEXLER discloses a block diagram illustrating an example room in which an end user obtains a digital image of sample window is shown in FIG. 3. The distance measurement mechanism enables the automatic measurement of dimensions for a window part from a digital image of the window that includes a reference object in substantially the same plane as the constraint dimension associated with the window. The end user 131 takes a photograph, using a digital image acquisition device such as digital camera 133, which includes a window 130 and the reference object 132. Knowledge of the reference object dimensions is used to calculate any dimensions needed for fabrication of one or more parts for fenestration (wherein reference objects and target objects may include walls, furniture, pictures, holes/openings in walls, etc.). In paragraph [0118]-WEXLER discloses in some cases it may be desirable to capture only a single image. In paragraph [0244]-WEXLER discloses once the reference object is located, the adhering objects are found in the digital image by color and/or shape segmentation of the image (step 541). In paragraph [0245]-WEXLER discloses an edge detector, such as the well-known Canny edge detector and probabilistic Hough transform may be used to create a consensus line within each region of interest (step 552). Please also read paragraph [0153-0157 and 0238]) to correct for image distortion to produce a geometrically transformed digital image (Fig. 26. Paragraph [0245]-WEXLER discloses once the preliminary perspective transform is obtained, it is adjusted so that the center point and center geometric mean scale of the original image is preserved such that the aspect ratio of the reference object is maintained. This is achieved by balancing the relative scaling of the 2D transform (step 560), normalizing the perspectivity matrix (step 562) and preserving the image center location (step 564). Please also read paragraph [0153-0163]), wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object (Fig. 2. Paragraph [0157]-WEXLER discloses after the transform is determined, repeated correction of the perspective distortion in the triage resolution image is optional. In paragraph [0167]-WEXLER discloses fiducial size differences relative to the reference object (step 420) may be used to make adjustments needed to account for out-of-plane scale distortions induced by the use of the pane plane projection transform and may be used to calculate corrected pixels per inch for the plane of interest to allow for more accurate measurement of such dimensions (wherein perspective transformation is a form of projective transformation). Please also read paragraph [0155, 0169, 0172, 0180]); and
applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object (Fig. 26. Paragraph [0246]-WEXLER discloses a flow diagram illustrating an example method for calculating target object dimensions is shown in FIG. 27. The self-adhering pane ancillary objects are re-found in the corrected/transformed image (step 570). The pixel dimensions of the transformed bounding box 465 created from the pane ancillary objects are then determined (step 574). The pixel dimensions are then converted to the pane physical dimensions using the reference object pixel scale factor (i.e. pixels per unit length) (step 576). In paragraph [0247]-WEXLER discloses the method for finding ancillary objects illustrated in FIG. 25 may also be used to find ancillary objects used to define non-transparent target objects such as walls or portions thereof. In such cases, an ancillary object edge adhered to the wall may be used to define a wall corner, the edge defined where the wall and floor or the wall and ceiling meet. Please also read paragraph [0154-0160, 0177-0180]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, wherein said standard construction object is an electrical wall plate selected from the group consisting of an electrical outlet wall plate, a toggle switch wall plate, and a paddle switch wall plate; determining dimensions of the detected object; applying image segmentation to the single digital image to produce a segmented image, with the teachings of WEXLER of having applying geometric transformation, field of view and depth correction to the combined output image to correct for image distortion to produce a geometrically transformed digital image, wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object; and applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan.
Wherein having AARTS method having applying geometric transformation, field of view and depth correction to the combined output image to correct for image distortion to produce a geometrically transformed digital image, wherein said applying geometric transformation comprises applying projective transformation to the single digital image to correct for image distortion caused from perspective, shear, and/or stretch of the detected object; and applying dimensions to the geometrically transformed digital image at least partially based on the dimensions of the detected object to produce a dimensionalized floorplan.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and WEXLER concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while WEXLER provides systems and methods that processes one or more digital images to provide improved measurement capabilities. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and WEXLER et al. (US 20170284799 A1), Abstract, Paragraph [0045].
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over AARTS et al. (US 20120183204 A1), hereinafter referenced as AARTS in view of FATHI (US 20160364885 A1), hereinafter referenced as FATHI and in further view of WEXLER et al. (US 20170284799 A1), hereinafter referenced as WEXLER and in further view of GAUSEBECK et al. (US 20190026958 A1), hereinafter referenced as GAUSEBECK.
Regarding claim 3, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 2, AARTS in view of FATHI fail to explicitly teach further comprising using square footage of the floorplan while generating the three-dimensional construction image.
However, GAUSEBECK explicitly teaches further comprising using square footage of the floorplan while generating the three-dimensional construction image (Fig. 1. Paragraph [0080]- GAUSEBECK discloses measurement data (e.g., square footage, etc.) associated with surfaces can also be determined based on the derived 3D data corresponding to the respective surfaces and associated with the respective surfaces. These measurements can be displayed in association with viewing and/or navigation of the 3D floorplan model. Calculation of area (e.g., square footage) can be determined for any identified surface or portion of a 3D model with a known boundary by summing areas of polygons comprising the identified surface or the portion of the 3D model. Displays of individual items (e.g., dimensions) and/or classes of items can be toggled in a floorplan via a viewer on a remote device (e.g., via a user interface on a remote client device)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of WEXLER and in further view of FATHI of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of GAUSEBECK of having further comprising using square footage of the floorplan while generating the three-dimensional construction image.
Wherein having AARTS method having further comprising using square footage of the floorplan while generating the three-dimensional construction image.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and GAUSEBECK concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while GAUSEBECK provides systems and methods for predicting 3D data from 2D data using deep learning techniques that enhances perception accuracy. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and GAUSEBECK et al. (US 20190026958 A1), Abstract and Paragraph [0123, 0132, 0160 and 0171].
Claims 4 is rejected under 35 U.S.C. 103 as being unpatentable over AARTS et al. (US 20120183204 A1), hereinafter referenced as AARTS in view of FATHI (US 20160364885 A1), hereinafter referenced as FATHI and in further view of WEXLER et al. (US 20170284799 A1), hereinafter referenced as WEXLER and in further view of LORENZO (US 20190327413 A1), hereinafter referenced as LORENZO.
Regarding claim 4, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS in view of FATHI fails to explicitly teach wherein the dimensionalized floorplan is wireframe image data.
However, LORENZO explicitly teaches wherein the dimensionalized floorplan (Fig. 7. Paragraph [0073]-LORENZO discloses referring now to FIG. 7, a flow diagram illustrating non-panoramic image transfer. FIG. 7 shows the process flow between a non-360 degree image capture device 116, an image processing device 600, and a 2D floor plan and 3D model 704. In paragraph [0076]-LORENZO discloses at block 724, the non-360 degree image capture device 116 captures one or more non-360 degree images. Non-360 degree images 724 may be transferred one at a time as captured images 728. In paragraph [0077]-LORENZO discloses the image processing device 600 then extracts 2D X-Y coordinates and orientations from the received image or images 728. In paragraph [0078]-LORENZO discloses once the 2D coordinates and orientations have been determined 732, the X-Y coordinates are converted into 3D model coordinates 736) is wireframe image data (Fig. 7. Paragraph [0079]-LORENZO discloses next, the captured images and model viewpoints are uploaded to the 2D floor plan 744 at the corresponding 2D X-Y coordinates. Please also see Fig. 3 and 5 and read paragraph [0074]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of LORENZO of having wherein the dimensionalized floorplan is wireframe image data.
Wherein having AARTS method having wherein the dimensionalized floorplan is wireframe image data.
The motivation behind the modification would have been to obtain a method that allows for the generation, manipulation and annotation of 3D models, since both AARTS and LORENZO concern systems and methods for generating virtual environments from two dimensional images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while LORENZO provides systems and methods provide for the creation of annotations on building floor plans, which allows for quick comparisons between photographs and corresponding model viewpoints in 3D models of the same building. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and LORENZO (US 20190327413 A1), Abstract and Paragraph [0028].
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over AARTS et al. (US 20120183204 A1), hereinafter referenced as AARTS in view of FATHI (US 20160364885 A1), hereinafter referenced as FATHI and in further view of WEXLER et al. (US 20170284799 A1), hereinafter referenced as WEXLER and in further view of CROWLEY et al. (US 20200233901 A1), herein referenced as CROWLEY.
Regarding claim 7, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS in view of FATHI fails to explicitly teach wherein determining dimensions of the detected object is based upon identifying the detected object using a brand, serial number, model number or combinations thereof.
However, CROWLEY explicitly teaches wherein determining dimensions of the detected object is based upon identifying the detected object (Fig. 3. Paragraph [0049]-CROWLEY discloses embodiments described herein can identify a physical object both when the object is static and additionally, or alternatively, when the object is in motion. These and other embodiments can also dimension the object either statically or in motion. In paragraph [0085]-CROWLEY discloses FIG. 2 is a set of exemplary object records arranged to store, retrieve, and process identification information, dimension information, and optional additional information. These physical objects have size, weight, and other qualities that exist in the physical world (wherein digital image data may be derived from two-dimensional data, three-dimensional data, video data, composite data from several images, etc. and dimension information may include actual measurement data, such as linear measurement units (e.g., inches, meters, or the like), volumetric measurement units (e.g., cubic inches (in.sup.3), cubic meters (m.sup.3), or the like), units of weight (e.g., pounds, kilograms, or the like), units of flexibility or stiffness (e.g., feet/pound, meters/Newton, or the like), etc.)) using a brand, serial number, model number or combinations thereof (Fig. 3. Paragraph [0092]-CROWLEY discloses the additional information may include, for example, a description, manufacturer, model number, serial number).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of CROWLEY discloses of having wherein determining dimensions of the detected object is based upon identifying the detected object using a brand, serial number, model number or combinations thereof.
Wherein having AARTS method having wherein determining dimensions of the detected object is based upon identifying the detected object using a brand, serial number, model number or combinations thereof.
The motivation behind the modification would have been to obtain a method that allows for the generation and manipulation of 3D models and improved measurement results, since both AARTS and CROWLEY discloses concern systems and methods for processing two/three dimensional images and dimensionalizing objects. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while CROWLEY discloses provides systems and methods that improve computer-based systems that act on physical objects by providing access to both identification information and dimension information. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and WEXLER (US 20200233901 A1), Abstract, Paragraph [0008].
Claims 12 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over AARTS et al. (US 20120183204 A1), hereinafter referenced as AARTS in view of WEXLER (US 20160104288 A1), hereinafter referenced as WEXLER and in further view of FATHI (US 20160364885 A1), hereinafter referenced as FATHI and in further view of Buzz et al. (US 20140244338 A1), herein referenced as Buzz.
Regarding claim 12, AARTS in view of FATHI and in further view of WEXLER explicitly teach the method of claim 1, AARTS further teaches further comprising: identifying a reference dimension within the digital image (Fig. 3. Paragraph [0027]-AARTS discloses in step 306, information concerning the captured image is gathered by the information gathering unit 112. Please also read [0028]);
calculating the length of the reference dimension (Fig. 3. Paragraph [0028]-AARTS discloses in step 310, the image analysis unit 114 identifies objects in the image. The image analysis unit 114 may identify objects in an image by analyzing the pixels in the image to determine lines where the pixel colors change from one color to another. The image analysis unit 114 may also identify objects by comparing areas identified in the image to a database of known images);
converting the length of the reference dimension into a pixel equivalency dimension (Fig. 3. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room. The image conversion unit 116 generates a three dimensional plane for each wall of the room, and stores these planes in the memory 210. In addition, the image conversion unit 116 converts each object in the room into a three dimensional object by relating the dimensions of each object to the dimensions of the room and the position of each object within the room. Further in paragraph [0043]- AARTS discloses the image analysis unit 114 may consistently adjust the lengths of walls and other object dimensions to ensure the accuracy of the image is maintained. Please also read paragraph [0055]);
AARTS fails to explicitly teach and using the reference dimension to produce a three-dimensional context-rich takeoff package.
However, Buzz explicitly teaches and using the reference dimension to produce a three-dimensional (Fig. 1. Paragraph [0014]-Buzz discloses the Image tab 108 provides a view of a construction drawing, such as the construction drawing 120. Further in paragraph [0017]-Buzz discloses the two dimensional image shown in the drawing 120 can also be three dimensional (3D) such as a 3D CAD drawing, Building Information Model (BIM), or the like)) context-rich takeoff package (Fig. 1. Paragraph [0014]-Buzz discloses a dropdown menu 104 allows the user to select which page or area of the construction project to view. The selected area can be any part of the construction project for which a separate construction drawing exists. For example, the dropdown menu 104 in screen 100 shows that the 2.sup.nd Floor Plan has been selected. As such the corresponding drawing 120 includes all the various building conditions in the 2.sup.nd Floor area of this construction project).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of Buzz of having wherein the three-dimensional context-rich takeoff package comprises a three-dimensional construction image and a corresponding materials takeoff list.
Wherein having AARTS method having wherein the three-dimensional context-rich takeoff package comprises a three-dimensional construction image and a corresponding materials takeoff list.
The motivation behind the modification would have been to obtain a method that improves the accuracy for generating 3D models, since both AARTS and Buzz concern systems and methods for generating, displaying and analyzing three dimensional images in the context of building. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while Buzz’s systems and methods allow for a detailed takeoff packages based on multiple building conditions to be produced for a three-dimensional digital representation of a construction project. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and Buzz et al. (US 20140244338 A1), Abstract and Paragraph [0014-0015 and 0017].
Regarding claim 14, AARTS in view of WEXLER and in further view of FATHI and in further view of BUZZ explicitly teach the method of claim 12, AARTS further teaches wherein pixel walking is used calculating the length of the reference dimension (Fig. 5. Paragraph [0036]-AARTS discloses FIG. 5 illustrates a process of determining the dimensions of a room from an image 400. In step 502, the information gathering unit 112 presents the image 400 to a user via the GUI 232 on the client device 104/106. In step 504, the information gathering unit 114 receives basic dimensional information of the image via the GUI 232, and from the information stored in the image such as EXIF information (wherein dimensional information may include the height h of the room 400 depicted in the image 400, the angles a and b between walls 402, 404 and 406, the length of lines that form intersections between walls, ceiling, and the floor 408, a depth dimension, such as the width of an object or feature on the walls). The image analysis unit 114 may also use a line analysis algorithm to identify where the lines that forms the intersections between the walls (wherein the line analysis algorithm may include a Hough transform algorithm). Further in paragraph [0047]-AARTS discloses in step 612, the image analysis unit 114 identifies objects 604 within the removal area 602. The image analysis unit 114 may use any known object identification technique such as edge detection. In paragraph [0048]-AARTS discloses in step 614, the image analysis unit 114 divides the removal area 602 into target patches. In paragraph [0049]-AARTS discloses in step 616, the image analysis unit 114 identifies the traversal order of the target patches in the removal area 602. The target patch traversal order may be based on the amount of pixel information available on the borders of each target patch).
Regarding claim 15, AARTS in view of FATHI and in further view of WEXLER and in further view of BUZZ explicitly teach the method of claim 12, AARTS fails to explicitly teach wherein the three-dimensional context-rich takeoff package comprises a three-dimensional construction image and a corresponding materials takeoff list.
However, Buzz explicitly teaches wherein the three-dimensional context-rich takeoff package (Fig. 1. Paragraph [0014]-Buzz discloses a dropdown menu 104 allows the user to select which page or area of the construction project to view. The selected area can be any part of the construction project for which a separate construction drawing exists. For example, the dropdown menu 104 in screen 100 shows that the 2.sup.nd Floor Plan has been selected. As such the corresponding drawing 120 includes all the various building conditions in the 2.sup.nd Floor area of this construction project) comprises a three-dimensional construction image (Fig. 1. Paragraph [0014]-Buzz discloses the Image tab 108 provides a view of a construction drawing, such as the construction drawing 120. Further in paragraph [0017]-Buzz discloses the two dimensional image shown in the drawing 120 can also be three dimensional (3D) such as a 3D CAD drawing, Building Information Model (BIM), or the like)) and a corresponding materials takeoff list (Fig. 5A-B and 6, #116 called a conditions list. Paragraph [0034]-Buzz discloses in the screen 400 of FIG. 5A, four building conditions have been selected. These include 24''.times.24'' Ceramic Tile Floor, Ceramic Tile Base, Paint Walls--Epoxy 9', and Paint GWB Ceilings).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER and in further view of Buzz of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of Buzz of having wherein the three-dimensional context-rich takeoff package comprises a three-dimensional construction image and a corresponding materials takeoff list.
Wherein having AARTS method having wherein the three-dimensional context-rich takeoff package comprises a three-dimensional construction image and a corresponding materials takeoff list.
The motivation behind the modification would have been to obtain a method that improves the accuracy for generating 3D models, since both AARTS and Buzz concern systems and methods for generating, displaying and analyzing three dimensional images in the context of building. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while Buzz’s systems and methods allow for a detailed takeoff packages based on multiple building conditions to be produced for a three-dimensional digital representation of a construction project. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and Buzz et al. (US 20140244338 A1), Abstract and Paragraph [0014-0015 and 0017].
Claim 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over AARTS et al. (US 20120183204 A1), hereinafter referenced as AARTS in view of FATHI (US 20160364885 A1), hereinafter referenced as FATHI and in further view of WEXLER et al. (US 20170284799 A1), hereinafter referenced as WEXLER and in further view of Buzz et al. (US 20140244338 A1), herein referenced as Buzz and in further view of HALLIDAY et al. (US 20170132835 A1), hereinafter referenced as HALLIDAY.
Regarding claim 13, AARTS in view of FATHI and in further view of WEXLER and in further view of Buzz explicitly teach the method of claim 12, AARTS fails to explicitly teach further comprising validating the pixel equivalency dimension by using objects of standard dimensions.
However, Halliday explicitly teaches further comprising validating the pixel equivalency dimension by using objects of standard dimensions. (Fig. 3. Paragraph [0035]-Halliday discloses in step 303, perimeter boundaries for the identified architectural element(s) are defined by correlating, for example, perimeter points, vertices, corner points, edges or specific salient pixels (e.g., neighboring pixels with noted changes in contrast, density or color, side pixels, corner pixels, etc.) of the defined architectural element within the ground-based image to the corresponding boundaries (represented by x, y, z positions) within the 3D building model (e.g., within two images having a common element). Pixel positions are extrapolated from vertices/edges of the ground-level image. Further in paragraph [0036]-Halliday discloses in step 304, dimensional ratios of distances spanning the width and length of the identified architectural element are determined using, for example, image processing system 100 of FIG. 1. Additionally in paragraph [0037]-Halliday discloses in step 305, determined ratios are compared to known standard architectural element dimensional ratios (width-to-height, width-to-height or area). Moreover in paragraph [0041]-Halliday discloses in step 306, the ratio which is closest to a known standard architectural element dimensional ratio is used as a scale. And in step 308, the scale is used to scale/rescale a multi-dimensional (e.g., 2D/3D) building model)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER and in further view of BUZZ of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of HALLIDAY of having further comprising using square footage of the floorplan while generating the three-dimensional construction image.
Wherein having AARTS method having further comprising using square footage of the floorplan while generating the three-dimensional construction image.
The motivation behind the modification would have been to obtain a method that improves the accuracy for generating 3D models, since both AARTS and HALLIDAY concern systems and methods for generating virtual environments from images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while HALLIDAY provides systems and methods that enhance the accuracy of scaling when constructing a labeled and dimensioned multi-dimensional (e.g., 3D) building model from building object imagery. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and HALLIDAY et al. (US 20170132835 A1), Abstract and Paragraph [0031, 0040 and 0042].
Regarding claim 18, AARTS in view of WEXLER and in further view of FATHI explicitly teach the method of claim 17, although AARTS explicitly teaches the single 2D pixel image (Fig. 4. Paragraph [0029]-AARTS discloses in step 316, the image conversion unit 116 converts the image from a two dimensional image into a three dimensional image using the dimensions of the room).
AARTS in view of WEXLER fails to explicitly teach wherein the single 2D pixel image comprises a blueprint.
However, HALLIDAY explicitly teaches wherein the single pixel image comprises a blueprint (Fig. 4. Paragraph [0097]-HALLIDAY discloses referring back to step 418 (FIG. 4), a 3D blue print with dimensions and/or labels is created. The 3D blueprint can be returned to the capture device (e.g., smartphone) to be displayed to the user or be displayed to a potential homeowner as part of a remodeling or repair proposal (estimate)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of AARTS in view of FATHI and in further view of WEXLER and in further view of BUZZ of having a method implemented with instructions executed by a processor, comprising: processing a single digital image of an interior space; identifying at least one detected object within the single digital image, wherein said at least one detected object comprises a standard construction object having standard known dimensions, with the teachings of HALLIDAY of having wherein the single pixel image comprises a blueprint.
Wherein having AARTS method having wherein the single 2D pixel image comprises a blueprint.
The motivation behind the modification would have been to obtain a method that improves the accuracy for generating 3D models, since both AARTS and HALLIDAY concern systems and methods for generating virtual environments from images. Wherein AARTS provides systems and methods for generating and manipulating an accurate and scaled virtual environment from two dimensional images, while HALLIDAY provides systems and methods that enhance the accuracy of scaling when constructing a labeled and dimensioned multi-dimensional (e.g., 3D) building model from building object imagery. Please see AARTS et al. (US 20120183204 A1), Abstract and Paragraph [0019, 0032 and 0055] and HALLIDAY et al. (US 20170132835 A1), Abstract and Paragraph [0031, 0040 and 0042].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant`s disclosure.
HARVIAINEN et al. (US 20190220954 A1)- Apparatus, method, and computer program code for producing composite image. A method comprises: obtaining (150) a first image (130) with first depth information; obtaining (152) a second image (132) with second depth information; detecting (154) regions of interest in the first image (130); segmenting (156) the regions of interest into individual layers; performing (158) image compensation for the individual layers according to the first depth information so that in image compensated individual layers a perspective and/or a viewpoint of the first image (130) is modified to be more similar with the second image (132); and rendering (160) the image compensated individual layers with the second image (132) utilizing the first depth information and the second depth information in order to produce a composite image (134).....................Please see Fig. 2 and para. [0050-0065]. Abstract.
McNAMER et al. (US 20110025825 A1)- Methods, systems, and computer program products for generating three-dimensional images of a scene are disclosed herein. According to one aspect, a method includes receiving a plurality of images of a scene. The method also includes determining attributes of the plurality of images. Further, the method includes determining, based on the attributes, a pair of images from among the plurality of images for use in creating a three-dimensional image. A user can, by use of the subject matter disclosed herein, use an image capture device for capturing a plurality of different images of the same scene and for converting the images into a three-dimensional, or stereoscopic, image of the scene. The subject matter disclosed herein includes a three-dimensional conversion process. The conversion process can include identification of suitable pairs of images, registration, rectification, color correction, transformation, depth adjustment, and motion detection and removal.....................Please see Fig. 3A. Abstract.
COHEN et al. (US 20170330388 A1)- Systems and methods for estimating a layout of a room are disclosed. The room layout can comprise the location of a floor, one or more walls, and a ceiling. In one aspect, a neural network can analyze an image of a portion of a room to determine the room layout. The neural network can comprise a convolutional neural network having an encoder sub-network, a decoder sub-network, and a side sub-network. The neural network can determine a three-dimensional room layout using two-dimensional ordered keypoints associated with a room type. The room layout can be used in applications such as augmented or mixed reality, robotics, autonomous indoor navigation, etc...................Please see Fig. 1. Abstract.
Lee et al. (US 20180268220 A1)- Systems and methods for estimating a layout of a room are disclosed. The room layout can comprise the location of a floor, one or more walls, and a ceiling. In one aspect, a neural network can analyze an image of a portion of a room to determine the room layout. The neural network can comprise a convolutional neural network having an encoder sub-network, a decoder sub-network, and a side sub-network. The neural network can determine a three-dimensional room layout using two-dimensional ordered keypoints associated with a room type. The room layout can be used in applications such as augmented or mixed reality, robotics, autonomous indoor navigation, etc...................Please see Fig. 1A-C. Abstract.
Tang et al. (US 11551422 B2)-Various implementations disclosed herein include
devices, systems, and methods that generate floorplans and measurements using a three-dimensional (3D) representation of a physical environment generated based on sensor data..................Please see Fig. 4-8. Abstract.
PATE et al. (US 20120259743 A1)- The disclosed system and method provide the user with the capability of selecting from among various particular products utilized in the design of an interior space, such as a kitchen, reviewing product specifications, design and finish combinations, and visualizing the products, both in isolation and placed in a photorealistic depiction of the selected products in one of a selected number of different room layouts. The system provides the user with the option of creating a unique account, including contact information, and information relating to one or more product selections preferred by the customer, which may then be electronically transmitted by the customer to a selected location remote from the system. The system and method may also provide the capability for a limited group, such as product manufacturer and/or dealer personnel, to access one or more of the customers' unique accounts to obtain contact information and/or product preferences for the customers....................Please see Fig. 4-5 and 12. Abstract.
LI et al. (US 20210064216 A1)- Techniques are described for using computing devices to perform automated operations involved in analysis of images acquired in a defined area, as part of generating mapping information of the defined area for subsequent use (e.g., for controlling navigation of devices, for display on client devices in corresponding GUIs, etc.). The defined area may include an interior of a multi-room building, and the generated information including a floor map of the building, such as from an analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees around a vertical axis)—the generating may be further performed without detailed information about distances from the images' viewing locations to objects in the surrounding building...................Please see Fig. 4-8. Abstract.
SHUSTER et al. (US 20170186228 A1)- An apparatus, method and system facilitate efficient creation of virtual places and provide tools for using the virtual places. The virtual places include a virtual real estate listing, newsworthy place and a virtual box seat. Tools are provided including an automatic declutter tool and a staging tool.…................. Please see Fig. 1, 3-4 and 5. Abstract.
Pitzer et al. (US 20140267717 A1)- A method for generating a floor plan of a room uses a mobile electronic device and a range finder to produce approximate and modified floor plans. The method includes receiving a plurality of strokes corresponding to walls in a room with a gesture input device, generating an approximate floor plan of the room based on the strokes, generating an approximate floor plan of the room with reference to the plurality of strokes, receiving an input gesture corresponding to one wall in the approximate floor plan of the room for measurement, receiving measurement data from a range finder corresponding to a dimension of the selected one wall, modifying the approximate floor plan with reference to the measurement data from the range finder, and generating with a display of the modified floor plan of the room...…................. Please see Fig. 2. Abstract.
NAGAR et al. (US 20160300293 A1)- Embodiments of the present disclosure are directed to methods, systems and devices for designing a commercial or residential space via a design application. For example, in some embodiments, a method is disclosed which enables a user to input information, including, for example, photos or video of the space, lighting, color(s). user preferences, measurements and the configuration and/or location of openings in the space. In such embodiments, the user can select a design theme, style or designer, and based on the information input (or acquired), the method presents recommendations of a new design for the space, which may include recommendations of products to furnish the space. Further embodiments also include enabling the user to purchase such products, and may also allow the user to hire service personnel to construct the recommended design and/or install selected/purchased products…................. Please see Fig. 1-6. Abstract.
CALMAN et al. (US 8668498 B2)- System, method, and computer program product are provided for using real-time video analysis, such as augmented reality, to assist a user of a mobile device with interior design. Through the use of real-time object recognition features, logos, artwork, products, locations, etc. can be recognized in a real-time video stream and can subsequently be matched with data associated with such to assist the user with selecting the proper design elements for a space, such as a kitchen. The proper design elements may be based off of several factors regarding the design space, include the dimensions of the space, the location of windows, doors and outlets, geographic and positional data, other space features, current design elements, architectural features, decor, and style data of the space, etc. This invention provides a virtual area with design elements, such that a user may view the area and determine the proper decor for the area...…................. Please see Fig. 4-6. Abstract.
Hodgkins et al. (US 20220121785 A1)-A system and method for determining material take-off from a 2D drawing is provided. A pre-processing component receives and pre-process drawings before they are categorised by a categoriser component by way of pre trained convolutional neural networks to determine the type of the processed image from one or more categories of drawing types. A material identifier component determines the probability that a feature in the processed image is present and an output component provides a unique identifier for each feature; a list of coordinates indicating the location of the feature on the processed image; and/or a list of coordinates describing the location of any text or other encoded information that is associated with the feature....................Please see Fig. 2-3. Abstract.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to Aaron Bonansinga whose telephone number is (703) 756-5380 The examiner can normally be reached on Monday-Friday, 9:00 a.m. - 6:00 p.m. ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Chineyere Wills-Burns can be reached by phone at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON TIMOTHY BONANSINGA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights