Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-22 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 11-16, 21 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Hur et al. (US 20210407142 A1) in view of Steinberg et al. (US 20190318177 A1) as cited in an 892.
Regarding claim 1, Hur teaches in a mobile computerized device, a method for generating visually coherent lighting for mobile augmented reality (generating a virtual hologram for a AR display on a mobile devices: para [0303-0304), [0311)), comprising:
capturing a set of near-field observations and a set of far-field observations of an environment (capturing AR environment/space data using distance based
partitioning of regions into close (near-field} and far (far-field} regions/bounding boxes: para [0091), [0304), [0332), [0354), (0412));
generating an environment map based upon the set of near-field observations of the environment and the set of far-field observations (adapting quantization parameters QP of region-wise point cloud groups based on distance so the near-field has higher
density than the far-field points; para [0163), [0347-0350), (0369-0370), (0412-0415)) of the environment (integrating spatial/environment coordinate content from the clouds; para [0086-0091), [0099)) […];
applying the environment map to a virtual object to render a visually-coherent virtual object (AR/MR content (virtual object) displaying objects; para [0091), [0209), [0311)) […]; and
displaying an image of the environment and the visually-coherent virtual object within the environment on a display of the mobile computerized device (para [0266-0268), [0303-0304), [0311)) but doesn’t explicitly disclose:
the environment map representing lighting of the environment; the visually-coherent virtual object comprising lighting corresponding to the lighting of the environment (The examiner notes the object referenced that has a point cloud representation is considered to be a virtual object. The ambient light level or reflectivity value is representative of the lighting of the environment ;para [311],[321), [330]).
Steinberg teaches the environment map representing lighting of the environment (The point cloud (environment map), may contain ambient light; para [311],[321), [330] ); the visually-coherent virtual object comprising lighting corresponding to the lighting of the environment (The examiner notes the object referenced that has a point cloud representation is considered to be a virtual object. The ambient light level or reflectivity value is representative of the lighting of the environment ;para [311],[321), [330]).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Hur in view Steinberg as doing so as real world lighting is complex an environment map stores the light from the entire surrounding sphere in a single image, making it a very efficient way to simulate these complex effects in real-time or during rendering.
Regarding claim 2, Hur in view Steinberg teaches the method of claim 1, wherein capturing the set of near-field observations and the set of far-field observations of an environment comprises: capturing a data frame of the environment by a camera system of the mobile computerized device (Hur para [0088));
applying a near field boundary to the data frame (bounding regions of a close frame; para [0332), [0353-0354));
defining a portion of the data frame within the near field boundary as a near-field observation (a extracting a close viewport region range; para [0215), [0332), [0334)); and
defining a portion of the data frame outside the near field boundary as a far-field observation (a extracting a greater distance viewport region range; para (0215),
[0332), [0334)).
Regarding claim 3, Hur in view Steinberg teaches the method of claim 2, wherein capturing the data frame of the environment by the camera system of the mobile computerized device comprises: capturing a first data frame of the environment by the camera system of the mobile computerized device (Hur para [0076), [0088));
identifying first position data associated with the mobile computerized device for the first data frame (capture 3D position metadata; para (0075),
(0090-0091), (0265), (0304));
capturing a second data frame of the environment by the camera system of the mobile computerized device (para (0076), [0088)};
identifying second position data associated with the mobile computerized device for the second data frame (para
(0075), [0090-0091), [0265), [0304));
comparing the first position data with the second position data (analyzing neighboring points within a
radius; para (0128), (0160)); and
when a difference between the second position data and the first position data is greater than a position threshold, discarding the second data frame (removing/separating groups of points outside the radius/partitioning range; Hur para (0103), [0105), (0128), [0168), [0353)).
Regarding claim 4, Hur in view Steinberg teaches the method of claim 2, comprising: for each near-field observation, identifying depth image data and color image data for each data point of the near-field observation (3D depth and color data;
para [0347), [0367), [0428)); and
for each near-field observation, applying device position data of the near-field observation to the depth image data and the color image data for each data point to construct a dense point cloud element of the dense point cloud (level of detail
for the depth and color is more dense; Hur para [0163), [0332), [0347), [0429)).
Regarding claim 5, Hur in view Steinberg teaches the method of claim 4, comprising aggregating the dense point cloud element of each near-field observation to generate a multi-view dense point cloud associated with the environment (grouping the dense points into a partial viewport of the environment rendered based on interactive head
orientation; Hur para [0070), [0079-0080), [0099), [0163)).
Regarding claim 6, Hur in view Steinberg teaches the method of claim 4, comprising: for each far-field observation, identifying color image data for each data point of the far- field observation (the level of color detail adjusted according to the far distance;
Hur para [0159), (0163), [0170-0171)); and
for each far-field observation, applying device position data of the far-field observation to the color image data for each data point to construct a sparse point cloud element of the sparse point cloud (construct a sparse point cloud with the
color; para (0163), (0170-0171), [0347), (0429)).
Regarding claim 21, Hur in view Steinberg teaches the method of claim 1, wherein the visually-coherent virtual object further comprises a reflective surface corresponding to the lighting of the environment (The examiner notes the object referenced that has a point cloud representation is considered to be a virtual object. The ambient light level or reflectivity value is representative of the lighting of the environment; Steinberg para [311],[321), [330]).
Claims 11-16 and 22 recites similar limitations to that of claims 1-6 and 21 and thus are rejected under similar rationale as detailed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7-10 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Hur et al. (US 20210407142 A1) in view of Steinberg et al. (US 20190318177 A1) in further view of Bradski et al. (US 20160026253 A1) as cited in an IDS.
Regarding claim 7, Hur in view Steinberg teaches the method of claim 6, comprising: projecting each sparse point cloud element of the sparse point cloud onto a set of anchor points on a unit sphere (reconstruction prediction transforming (projecting) sparse points onto point of an overlapped 3D coordinate (unit) space; para (0102), [0163), (0169-0170), [0428));
sampling each dense point cloud associated with the near-field observation to generate a set of sampled point clouds (para (0152)); and applying each sampled point cloud of the set of sampled point clouds onto the unit space as a set of sampled distributed points to generate a unit-space point cloud associated with the environment (para [0102), (0163), (0169-0170), (0320), (0336)), but Hur in view Steinberg fails to explicitly disclose fails to disclose a set of anchor points on a unit sphere; and applying each sampled point cloud of the set of sampled point clouds onto the unit sphere as a set of sampled distributed points to generate a unit-sphere point cloud.
Bradski teaches a set of anchor points on a unit sphere (locking anchor points to spherical coordinates; para [0670), (0691), (0799), (01648-1653)); and applying each
sampled point cloud of the set of sampled point clouds onto the unit sphere as a set of sampled distributed points to generate a unit-sphere
point cloud (registering ingested (sampled) point clouds with the optimal density (distribution) of points onto a sphere coordinate map to
advantageously render AR content; para [0582), (0670-0671), [0775), [0932)).
It would have been obvious to one of ordinary skill before the relevant date to modify the invention of Hur in view Steinberg to include a set of anchor points on a unit sphere; and applying each sampled point cloud of the set of sampled point clouds onto the unit sphere as a set of sampled distributed points to generate a unit-sphere point cloud as taught by Bradski for the advantage of providing more advantageous content projection.
Regarding claim 8, Hur in view Steinberg in further view of Bradski teaches the method of claim 7, wherein generating the environment map based upon the set of near-field observations of the environment and the set of far-field observations of the environment comprises:
performing a multi-resolution projection on a multi-view dense point cloud to apply to the environment map (projecting adjusted quantization density/level of detail points on an interactive viewport point cloud; para (0142), [0211-0212), (0276), (0365-0366)),; and Hur in view Steinberg doesn’t explicitly disclose:
performing an anchor extrapolation on the unit-sphere point cloud to apply to the environment map.
Bradski teaches performing an anchor extrapolation on the unit-sphere point cloud to apply to the environment map (locking on the sphere point cloud points referenced in the AR view; para (0669-0671), (0932)).
It would have been obvious to one of ordinary skill before the relevant date to modify the invention of Hur in view Steinberg to include performing an anchor extrapolation on the
unit-sphere point cloud to apply to the environment map as taught by Bradski for the advantage of providing enhanced content positioning.
Regarding claim 9, Hur in view Steinberg in view of Bradski teaches the method of claim 8, wherein performing the multi-resolution projection on the multi- view dense point cloud comprises:
converting a position of each
point of the multi-view dense point cloud from a Cartesian coordinate system to a different type of coordinate system (transforming
positions from XYZ coordinates to various 3D spaces; para [0093-0095), [0102), (0126)); identifying a two-dimensional projection
coordinate of each point on the environment map based on the different type of coordinate system (para (0093-0095), (0102), [0126));
assigning a size value to the two-dimensional projection coordinate of each point on the environment map (partitioning a region size; para
[0132), [0428). [0431)); and projecting a point cloud color of each point of the different type of coordinate system to the corresponding
two-dimensional projection coordinate on the environment map (transform the color of the points; para (0076), (0207), (0236)), Hur in view Steinberg fails to explicitly disclose the spherical coordinate system.
Bradski teaches the spherical coordinate system (para [0670), [0691-0693)).
It would have been obvious to one of ordinary skill before the relevant date to modify the invention of Hur to include the spherical coordinate system as taught by Bradski for the advantage of providing enhanced content mapping.
Regarding claim 10, Hur in view of Bradski teaches the method of claim 8, wherein performing the anchor extrapolation on the unit-sphere point cloud comprises:
identifying a color associated with points of the unit point cloud (para (0126-0128), (0428-0429)); generating a set of
points based upon a weighted average of the identified colors of adjacent points (weighted average color for neighbor points; para (0112),
[0159), [0171)); and assigning a color of each point to a corresponding point on the environment map (para (0107-0108), [0428-0429)), Hur in view Steinberg fails to disclose performing the anchor extrapolation on the unit-sphere point cloud comprises: identifying a color associated with each anchor point of the set of anchor points of the unit sphere point cloud; generating a set of extrapolated anchor points based upon a weighted average of the identified colors of adjacent anchor points; and a color of each extrapolated anchor point.
Bradski teaches identifying a color associated with each anchor point of the
set of anchor points of the unit sphere point cloud (color blending the anchor points; para [0358), (0639-0640)); generating a set of
extrapolated anchor points based upon a weighted average of the identified colors of adjacent anchor points (extrapolate locked/anchor
points using blended (weighted average) colors to identify natural features; para [0358), [0639-0640), (1648-1650)); and a color of each
extrapolated anchor point (para [0579), (0639-0640), [0979)).
It would have been obvious to one of ordinary skill before the relevant date to
modify the invention of HUR to include performing the anchor extrapolation on the unit-sphere point cloud comprises: identifying a color
associated with each anchor point of the set of anchor points of the unit sphere point cloud; generating a set of extrapolated anchor points
based upon a weighted average of the identified colors of adjacent anchor points; and a color of each extrapolated anchor point as taught by BRADSKI for the advantage of providing better identifying image features.
Claims 17-20 recite similar limitations to that of claims 7-10 are rejected under similar rationale as detailed above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT J CRADDOCK whose telephone number is (571)270-7502. The examiner can normally be reached Monday - Friday 10:00 AM - 6 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona E Faulk can be reached on 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT J CRADDOCK/Primary Examiner, Art Unit 2618