DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 2, 6-8, 11 and 15-17 are objected to because of the following informalities:
Claims 2 and 11 recite the limitation “a window view image”. It should read “the window view image”.
Claims 6 and 15 recite the limitation “a distant view layer proportion”. It should read “the distant view layer proportion”.
Claims 7 and 16 recite the limitation “a close-range view layer distance”. It should read “the close-range view layer distance”.
Claims 8 and 17 recite the limitation “from view distances” and “a close-range view layer distance”. It should read “from the view distances” and “the close-range view layer distance”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3-5 and 12-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 3-5 and 12-14 recite the limitation "the plurality of groups of setting comprises". There is insufficient antecedent basis for this limitation in the claim.
In addition, under the assumption of being read as “a plurality of groups of settings”, the examiner still could not determine which limitation, such as a window view image, a distant view layer proportion, a close-range view later distance, an OAF, or a window view openness index, in the corresponding independent claim is associated with this plurality of groups of settings. Therefore, rendered these claims as being indefinite.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over BISWAS et al. (US 20210019897 A1), referred herein as BISWAS in view of Miao et al. (US 20190172238 A1), referred herein as Miao and Lee et al. (US 20220383585 A1), referred herein as Lee.
Regarding Claim 1, BISWAS in view of Miao teaches a method for quantifying and calculating window view openness indexes based on window view photos and view distances, the method comprising (BISWAS Abst: an approach for providing accurate real-world distance or depth information from a monocular image; [0038] FIG. 1 is a diagram of a system capable of providing real-world distance information from a monocular image, according to one embodiment. The embodiments described herein address the technical problem of estimating how far specific objects in a given image are relative to the camera; [0051] FIGS. 2A and 2B are diagram illustrating example views of a camera orientation with respect to a road):
generating a window view image by an image capturing device (BISWAS [0109] FIG. 14 is a diagram of a user interface (UI) 1401 for providing real-world distance information from a monocular image, according to one embodiment. As shown, the mapping platform 111 receives a user input 1403 (e.g., via touch) to select a road sign 1405 in a monocular image presented in the UI 1401. Based on the selection, the user requests real-world distance and depth information for the base of the road sign 1405. In response, the mapping platform 111 can calculate a corresponding distance and depth of the selected road sign 1405 on the ground plane according to the embodiments described herein to update the UI 1401 with distance and depth information);
BISWAS does not but Miao teaches
computing a distant view layer proportion (Miao [0070] the processor can measure the maximum depth of the scene. If the maximum depth is measurable, and not infinite, the processor can define all objects that are within 30 percent of the maximum depth is foreground objects, and all other objects as background objects);
BISWAS in view of Miao further teaches
measuring and computing a close-range view layer distance (BISWAS [0044] the system 100 described herein addresses the technical problems and limitations of traditional approaches to provide real-world distances or depths of objects (e.g., in meters));
computing a view distance-based openness adjustment factor (OAF) (Miao [0071] to separate the visual representation into the foreground and the background, the processor can determine an average distance between the foreground object and the background object. When the average distance between the foreground object and the background object is below a predetermined threshold, the processor can reclassify the background object into the visual representation of the foreground object. The predetermined threshold can be 2 m, or can be a percentage of the thickness of the foreground object); the variation of thickness of the foreground object is interpreted as adjustment factor; and
BISWAS does not but Lee teaches
computing a window view openness index (Lee Abst: the scene information may be used to generate a virtual space of voxels where the method then determines the occupancy of the voxel space by comparing a variety of measurements; [0057] The term “relatively static time period” means a period of time in which the majority of the objects in a scene do not move very much relative to the camera and their distance to the camera… As used with respect to this term, the majority may be at least 70%, 80%, 85%, 90%, 95%, or 100% of the objects in the scene in certain embodiments. As used with respect to this term, the objects movement relative to the camera may be less than 0.1%, 0.2%, 0.5%, 1%, 2%, 5%, or 10% of the objects distance to the camera in certain embodiments). The occupancy is interpreted as openness index. Higer occupancy reads on lower index.
Miao discloses multiple sensors can include a depth sensor, a conventional camera, and a motion tracker providing inputs to the artificial intelligence module, which is analogous to the present patent application.
It would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified BISWAS to incorporate the teachings of Miao, and apply the thickness of the foreground object into the method of for estimating a real-world depth information from a monocular image.
Doing so would be able to separate foreground and background objects from images, video, point clouds and/or motion tracking data.
Lee discloses systems and/or methods that may be used for determining scene information (for example, 3D scene information) using data obtained at least in part from a camera array, which is analogous to the present patent application.
It would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified BISWAS to incorporate the teachings of Lee, and apply the method for determining the occupancy of the voxel space into the method of for estimating a real-world depth information from a monocular image.
Doing so would be able to generate a 3D representation of a scene or, by rapidly repeating the process in real time, a 3D video data stream of the unfolding scene.
Regarding Claim 2, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the generating a window view image comprises defining a plurality of groups of setting for the window view image generation (Lee [0101] The camera control unit 230 actions camera control signals via control lines 234, 233, 232, 231 enabling adjustment of one or more components of the lens system 210: the aperture 220, the shutter 221 and the sensor 223. Such controls may be used to adjust one or more of the following: imaging parameters (such as gain), exposure times, black level offsets and filter settings).
Regarding Claim 3, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the plurality of groups of setting comprises a group of setting including orientation attributes including heading, pitch, and tilt (BISWAS [0047] generalized to any pose of the camera ((e.g., location, orientation or pointing direction, etc., which are automatically computed); [0048] 1. Real-world locations of objects/features depicted in the image (e.g., depth indicating a forward distance from the camera to the object/feature, and a horizontal or sideways distance from the camera to the object/feature); [0049] 2. Where the camera is moving towards (heading); and/or [0050] 3. Camera Pose (e.g., angles the camera makes with the X, Y & Z axes with an origin at the focal point or optical center of the camera) which can also be modelled as yaw pitch and roll of the camera).
Regarding Claim 4, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the plurality of groups of setting comprises a group of setting including view frustum attributes including field of view (FoV) (BISWAS [0054] the inputs to the system 100 include the estimates of camera height, camera focal length, camera field of view, and/or the segmented input image).
Regarding Claim 5, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the plurality of groups of setting comprises a group of setting including positions of the image capturing device in a (x, y, z) coordinate system (BISWAS [0078] In one embodiment, the vanishing point module 401 can denote the image coordinates of this mid-point 733 as x.sub.h,y.sub.h. Assuming that the initial estimated vanishing point 723 has the coordinate x.sub.v, y.sub.v, the vanishing point module 401 obtains a new or subsequent point 735 with the coordinate x.sub.h, y.sub.v on a horizontal line running through the initial estimated vanishing point 723 as shown in FIG. 7H. This point at coordinate x.sub.h, is then provided as the new or subsequent estimate of the vanishing point).
Regarding Claim 6, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the quantifying and computing a distant view layer proportion comprises extracting a proportion of a distant layer of the window view image as a basic factor to measure view openness (Miao [0033] Based on the inputs, the artificial intelligence module can segment the received image and/or video into a foreground image and a background image to produce portrait imagery by blurring the background image and/or video; [0070] the processor can have a predetermined range of distances that define the foreground and background, such as any object within 2 m from the light sensor is in the foreground, while any object beyond 2 m from the light sensor is the background).
Regarding Claim 7, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the measuring and computing a close-range view layer distance are performed by a user or by a computing system (BISWAS [0097] In other words, the distance module 405 can compute the horizontal distance, a depth, or a combination of the feature location based on image coordinate data corresponding the vanishing point ray, the center line ray, the feature ray, one or more angles derived therefrom, and a known pixel-wise distance of the monocular image).
Regarding Claim 8, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the OAF is summarized from view distances computed in the step of measuring and computing a close-range view layer distance (Miao [0071] to separate the visual representation into the foreground and the background, the processor can determine an average distance between the foreground object and the background object... if the thickness of the foreground object is 0.5 m, any object whose average distance to the foreground object is within 1 m distance from the foreground object is considered to be also foreground object). The average distance participate the summarized distance.
Regarding Claim 9, BISWAS in view of Miao and Lee teaches the method of claim 1, and further teaches wherein the window view openness index (WVOI) is calculated based on the proportion of distant view layer and the OAF from the close-range view layer distance (Lee [0148] At step Update Voxel 655 the votes accumulated for a camera image are reviewed. If the maximum number of recorded votes is below a threshold, then the voxel is considered empty space or hidden by some foreground object. The internal representation of the voxel is then updated with the state Empty and a null is recorded for the representative spectral data. If the maximum number of recorded votes is equal of exceeds the threshold, then the voxel is considered to contain an observable surface and the internal representation of the voxel is updated to have a state of Occupied and its spectral data is copied from the geometric median of the pixels from the constituent camera images; [0149] In an alternative embodiment the step Evaluate Similarity 640 calculates a probability that a voxel contains a surface and step Update Voxel 655 applies a threshold to the probability to determine the voxel state as occupied or unoccupied). The occupancy is determined by the voxel threshold of foreground object, therefore, is inherently determined by the thickness of the foreground object.
Regarding Claims 10-18, BISWAS in view of Miao and Lee teaches a computer-readable storage medium having stored therein program instructions that, when executed by a processor of a computing system, cause the processor to execute a method for quantifying and calculating window view openness indexes based on window view images and view distances (BISWAS Abst: an approach for providing accurate real-world distance or depth information from a monocular image; [0038] FIG. 1 is a diagram of a system capable of providing real-world distance information from a monocular image, according to one embodiment. The embodiments described herein address the technical problem of estimating how far specific objects in a given image are relative to the camera; [0007] According to another embodiment, a non-transitory computer-readable storage medium carries one or more sequences of one or more instructions which, when executed by one or more processors, cause, at least in part, an apparatus to determine a vanishing point of the monocular image captured by a camera; [0051] FIGS. 2A and 2B are diagram illustrating example views of a camera orientation with respect to a road):
The metes and bounds of the claim substantially correspond to the claimed limitations set forth in claims 1-9; thus they are rejected on similar grounds and rationale as their corresponding limitations.
Regarding Claim 19, BISWAS in view of Miao and Lee teaches a window view openness index quantifying and calculating system (BISWAS Abst: an approach for providing accurate real-world distance or depth information from a monocular image):
The metes and bounds of the claim substantially correspond to the claimed limitations set forth in claim 1; thus they are rejected on similar grounds and rationale as their corresponding limitations.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Samantha (Yuehan) Wang whose telephone number is (571)270-5011. The examiner can normally be reached Monday-Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Samantha (YUEHAN) WANG/
Primary Examiner
Art Unit 2617