DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-6, 9-15 and 18-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Cui et al. (US PG Pub. 20220171963).
Regarding claims 1, 10 and 19, Cui discloses an system (computing system 500 of fig. 4) comprising:
memory storing instructions (memory 404 of fig. 4); and
one or more processors (processor 402 of fig. 4) coupled to the memory (illustrated in fig. 4), wherein the instructions, when executed by the one or more processors being configured to:
determine, based on sensor data (para. 0012; system, or command center, or may enter an area based on its own analysis. For instance, the AAV may analyze imaging sensor data it collects), a set of candidate surfaces on which to project visual information for a target person within a vehicle (para. 0013; To determine candidate projection areas, the AAV may first determine a projection search space that is (1) within the field of view of the AAV's imaging sensors. (2) that is within the range of the AAV's projector, and (3) that is most visible to the target audience), the set of candidate surfaces (para. 0013; candidate projection areas may comprise contiguous areas within the space) residing in an interior of the vehicle (examiner in interpreting “interior of the vehicle” as intended use. The “system” is the same if it is in an autonomous aerial vehicle/AAV or the interior of a vehicle, determining candidate projection areas is the same.);
based on the sensor data, determine, for each candidate surface from the set of candidate surfaces (para. 0012; system, or command center, or may enter an area based on its own analysis. For instance, the AAV may analyze imaging sensor data it collects), a respective discernability of the visual information when projected onto each candidate surface that represents how well the visual information can be visually distinguished from the candidate surface (para. 0013; the optimal projection zone may be the largest contiguous area (e.g., the largest candidate projection zone) within the space that is most free of objects and para. 0054; step 330 may identify those contiguous areas that are greater than the minimum size and that are relatively free of irregularities that would interfere with a visual projection of the information data. For instance, candidate projection areas may be completely free of irregularities, or may meet a requirement that any irregularity or irregularities are less than a threshold measure of a level of irregularity. For instance, image processing of the at least one image may apply scores or values to sub-regions within the image(s) regarding surface/terrain variability in terms of surface level/height, color, contrast, reflectance, etc. and the processing system may select the color and/or contrast between text and images of the projection to optimize for the surface color(s) of a candidate projection area that is selected as the projection zone. It should also be noted that the plurality of candidate projection areas may include at least one of: a ground surface candidate projection area or a non-ground surface projection area (e.g., a vertical surface perpendicular to the ground, such as a wall, sloped surfaces, such as a pitched roof, a flat roof that may be visible to the at least one person, who may be on a taller surface nearby, or who may be on the roof itself, outdoor pavement on the top of a parking deck, etc.), the respective discernability being determined based on attributes of the candidate surface and the visual information (para. 0026; computing device or processing system, such as computing system 400 depicted in FIG. 4 and may be configured to provide one or more functions in support of examples of the present disclosure for identifying candidate projection areas and selecting a candidate projection area as a projection zone for projecting informational data from an autonomous aerial vehicle.);
based on the respective discernability of each candidate surface from the set of candidate surfaces, select a particular candidate surface from the set of candidate surfaces as a target surface for projecting the visual information (para. 0026), the particular candidate surface being associated with at least a threshold amount of discernability of the visual information when projected onto the particular candidate surface (para. 0054; step 330 may include an analysis of the at least one image, wherein the search is bounded by a minimum contiguous area size. In other words, contiguous areas smaller than the threshold may be excluded from consideration as a candidate projection area. Similarly, there may be a maximum size beyond which the processing system will not attempt to identify contiguous areas as candidate projection zones. In one example, step 330 may identify those contiguous areas that are greater than the minimum size and that are relatively free of irregularities that would interfere with a visual projection of the information data);
adjust (para. 0015; AAV adjusts the size of the projected visual information accordingly, and projects the visual information within the optimal projection zone), based on attributes of the particular candidate surface (para. 0054; step 330 may identify the plurality of candidate projection areas within the projection search space that may be identified at optional step 325. Each candidate projection area may comprise a contiguous area that is deemed to be visible to the at least one person (and in one example, that is within range of a projector of the AAV). In an example where the at least one person comprises a group of people, each candidate projection area may be determined to be visible to at least one of the people within the group (based upon the positions and orientations that may be determined at step 320). In one example, step 330 may include an analysis of the at least one image, wherein the search is bounded by a minimum contiguous area size), one or more attributes of the visual information to increase discernability of the visual information when projected on the particular candidate surface, wherein at least one of the one or more attributes of the visual information is not a brightness (the attribute in step 330 is that the candidate projection area be “at least visible to the at least one person”); and
send, to a projector on the vehicle, a message instructing the projector to project the visual information with the adjusted one or more attributes onto the particular candidate surface (para. 0059; the projector of the AAV, wherein the projector may simultaneously illuminate the at least one item of interest and project the informational data in the projection zone).
Regarding claims 2, 11 and 20, Cui discloses wherein the instructions also cause the one or more processors (402) are configured to: determine, based on the sensor data (para. 0012; system, or command center, or may enter an area based on its own analysis. For instance, the AAV may analyze imaging sensor data it collects), whether there are any obstructions within a projection path from the projector to the particular candidate surface (para. 0035; one of the plurality of candidate projection areas is selected as a projection zone based upon one or more factors, e.g., as compared to others of the plurality of candidate projection areas, such as: a number of people within the group of people 140 to whom the one of the plurality of candidate projection areas is deemed to be visible, a size of the one of the plurality of projection areas, a range of the one of the plurality of projection areas with respect to a projector of AAV 160, or a measure of at least one surface characteristic of the one of the plurality of candidate projection areas (e.g., textured, glossy, dark/light, color contrasts, shade contrasts, a number/quantity and/or size of obstruction(s) within the one of the plurality of candidate projection areas (where obstructions can include one or more members of the group of people 140 and/or one or more other people in an area (such as non-group members 145), as well as non-human objects)); and select the particular candidate surface as the target surface for projecting the visual information further based on a determination that there are no obstructions within the projection path from the projector to the particular candidate surface (para. 0037; in FIG. 1 that candidate projection areas 151 and 152 are both free of obstructions, do not have rough surface features, and may be approximately the same distance from AAV 160 (and its projector, e.g., one of the modules 164)).
Regarding claims 3 and 12, Cui discloses wherein the instructions also cause the one or more processors (402) are configured to: determine, based on the sensor data (para. 0012; system, or command center, or may enter an area based on its own analysis. For instance, the AAV may analyze imaging sensor data it collects), whether there are any obstructions within a projection path from the projector to the particular candidate surface (para. 0036; parts of the search space 157 may be deemed to have too many obstructions or non-smooth surface characteristics such that these parts of the search space 157); determine that there are one or more obstructions within the projection path from the projector to the particular candidate surface (para. 0037; candidate projection areas 151 and 152 are both free of obstructions, do not have rough surface features, and may be approximately the same distance from AAV 160 (and its projector, e.g., one of the modules 164)); in response to determining that there are one or more obstructions within the projection path from the projector to the particular candidate surface, select a different candidate surface from the set of candidate surfaces to project at least a portion of visual information; and project at least the portion of visual information onto the different candidate surface (para. 0036; AAV 160 may first identify areas that are relatively free of large objects or surface features that would interfere with a visual projection from AAV 160. For instance, in the example of FIG. 1, the search space 157 may include at least one member of the group of people 140, one of the non-group members 145, bushes 158, and a hydrant 159. These parts of the search space 157 may be deemed to have too many obstructions or non-smooth surface characteristics such that these parts of the search space 157 may be excluded from consideration as candidate projection areas. As such, candidate projection areas 151 and 152 may be the resulting candidate projection areas after screening and excluding other parts of the search space 157).
Regarding claims 4 and 13, Cui discloses wherein selecting a different candidate surface from the set of candidate surfaces to project at least a portion of visual information (para. 0010; identifying candidate projection areas and selecting a candidate projection area as a projection zone for projecting informational data from an autonomous aerial vehicle) comprises: determining, based on the sensor data, whether there are any obstructions within respective projection paths from the projector to additional candidate surfaces from the set of candidate surfaces (para. 0044; projection zone 221 may comprise the largest contiguous area within the imaging/sensing range 229 that is most visible to two members of a group being tracked by the AAV, and/or that is the most free of obstructions, has the least irregularities, etc.); determining that there are no obstructions within the respective projection path from the projector to the different candidate surface (para. 0044), the different candidate surface comprising one of the additional candidate surfaces; and selecting the different candidate surface as an additional target surface for projecting at least the portion of visual information based on the determination that there are no obstructions within the respective projection path from the projector to the different candidate surface and a determination a discernability of at least the portion of visual information when projected onto the different candidate surface exceeds a threshold (para. 0054; there may be a maximum size beyond which the processing system will not attempt to identify contiguous areas as candidate projection zones. In one example, step 330 may identify those contiguous areas that are greater than the minimum size and that are relatively free of irregularities that would interfere with a visual projection of the information data. For instance, candidate projection areas may be completely free of irregularities, or may meet a requirement that any irregularity or irregularities are less than a threshold measure of a level of irregularity.).
Regarding claims 5 and 14, Cui discloses wherein determining the respective discernability of the visual information when projected onto each candidate surface comprises: comparing one or more first attributes of the candidate surface with one or more second attributes of the visual information (para. 0035); and determining the respective discernability of the visual information when projected onto each candidate surface based on the comparing of one or more first attributes of the candidate surface with one or more second attributes of the visual information (para. 0035; the plurality of candidate projection areas is selected as a projection zone based upon one or more factors, e.g., as compared to others of the plurality of candidate projection areas, such as: a number of people within the group of people 140 to whom the one of the plurality of candidate projection areas is deemed to be visible, a size of the one of the plurality of projection areas, a range of the one of the plurality of projection areas with respect to a projector of AAV 160, or a measure of at least one surface characteristic of the one of the plurality of candidate projection areas (e.g., textured, glossy, dark/light, color contrasts, shade contrasts, a number/quantity and/or size of obstruction(s) within the one of the plurality of candidate projection areas (where obstructions can include one or more members of the group of people 140 and/or one or more other people in an area (such as non-group members 145), as well as non-human objects), etc.). In one example, multiple factors may be weighted and combined to derive a score for each of the plurality of candidate projection areas, with a candidate projection area having the best score (e.g., highest or lowest) being selected as the projection zone).
Regarding claims 6 and 15, Cui discloses wherein the one or more first attributes and the one or more second attributes comprise at least one of color, texture, brightness levels, one or more dimensions, and one or more visible patterns (para. 0035; the plurality of candidate projection areas is selected as a projection zone based upon one or more factors, e.g., as compared to others of the plurality of candidate projection areas, such as: a number of people within the group of people 140 to whom the one of the plurality of candidate projection areas is deemed to be visible, a size of the one of the plurality of projection areas, a range of the one of the plurality of projection areas with respect to a projector of AAV 160, or a measure of at least one surface characteristic of the one of the plurality of candidate projection areas (e.g., textured, glossy, dark/light, color contrasts, shade contrasts).
Regarding claims 9 and 18, Cui discloses wherein the instructions also cause the one or more processors (402) are configured to:
determine, based on the sensor data (para. 0012; system, or command center, or may enter an area based on its own analysis. For instance, the AAV may analyze imaging sensor data it collects), whether there are any obstructions within a projection path from the projector to the particular candidate surface (para. 0036; parts of the search space 157 may be deemed to have too many obstructions or non-smooth surface characteristics such that these parts of the search space 157);
determine that there are one or more obstructions within the projection path from the projector to the particular candidate surface (para. 0036);
in response to determining that there are one or more obstructions within the projection path from the projector to the particular candidate surface,
determine whether there are any obstructions within an additional projection path from an additional projector within the vehicle to the particular candidate surface (para. 0036; AAV 160 may first identify areas that are relatively free of large objects or surface features that would interfere with a visual projection from AAV 160. For instance, in the example of FIG. 1, the search space 157 may include at least one member of the group of people 140, one of the non-group members 145, bushes 158, and a hydrant 159. These parts of the search space 157 may be deemed to have too many obstructions or non-smooth surface characteristics such that these parts of the search space 157 may be excluded from consideration as candidate projection areas. As such, candidate projection areas 151 and 152 may be the resulting candidate projection areas after screening and excluding other parts of the search space 157);
based on a determination that there are no obstructions within the additional projection path from the additional projector within the vehicle to the particular candidate surface (para. 0037; candidate projection areas 151 and 152 are both free of obstructions, do not have rough surface features, and may be approximately the same distance from AAV 160 (and its projector, e.g., one of the modules 164)), select the additional projector (modules 164 one or more projectors) to project at least one of the visual information and additional visual information onto the particular candidate surface (para. 0054; step 330 may include an analysis of the at least one image, wherein the search is bounded by a minimum contiguous area size. In other words, contiguous areas smaller than the threshold may be excluded from consideration as a candidate projection area. Similarly, there may be a maximum size beyond which the processing system will not attempt to identify contiguous areas as candidate projection zones. In one example, step 330 may identify those contiguous areas that are greater than the minimum size and that are relatively free of irregularities that would interfere with a visual projection of the information data); and
project the at least one of the visual information and additional visual information onto the particular candidate surface (para. 0059; the projector of the AAV, wherein the projector may simultaneously illuminate the at least one item of interest and project the informational data in the projection zone).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7, 8, 16 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cui et al. (US PG Pub. 20220171963) as applied to claim 1 above, and further in view of Kursula et al. (US PG Pub. 20180007328).
Regarding claims 7 and 16, Cui discloses a projection system (para. 0028; one or more module(s) 164 with one or more additional controllable components, such as one or more: microphones, loudspeakers, infrared, ultraviolet, and/or visible spectrum light sources, projectors) wherein the instructions also cause the one or more processors are configured to: determine, based on the sensor data, project at least one of the visual information and additional visual information onto the different candidate surface (para. 0026; computing device or processing system, such as computing system 400 depicted in FIG. 4 and may be configured to provide one or more functions in support of examples of the present disclosure for identifying candidate projection areas and selecting a candidate projection area as a projection zone for projecting informational data from an autonomous aerial vehicle).
Cui fails to teach wherein the instructions also cause the one or more processors are configured to: determine, an eye gaze of the target person; based on the eye gaze of the target person, determine that the target person is looking at a different candidate surface from the set of candidate surfaces; and based on the determining that the target person is looking at the different candidate surface.
Kursula discloses a viewpoint adaptive image projection system wherein the instructions also cause the one or more processors (viewpoint adapter 130 can be a processor of para. 0025) are configured to: determine, an eye gaze of the target person (para. 0029; the viewpoint adapter 130 can determine a gaze vector or gaze vectors); based on the eye gaze of the target person (para. 0031; perspective views of the projection surface, an example user 101, and an example adjusted projected images corresponding to different gaze vectors determined based on identification of the user in an image of the scene), determine that the target person is looking at a different candidate surface from the set of candidate surfaces; and based on the determining that the target person is looking at the different direction (para. 0031; FIG. 2 depicts the projection surface 150 and an adjusted projected image 200 corresponding to the gaze vector 164-1 while FIG. 3 depicts the projection surface 150 and an adjusted projected image 300 corresponding to the gaze vector 164-2. It is noted, that FIGS. 2-3 depict adjusted projected images 200 and 300 corresponding to different viewpoints (e.g., examples where a user moves positions, or the like). FIG. 4 depicts the projection surface 150 and an adjusted projected image 400 corresponding to the gaze vector 164-11 while FIG. 5 depicts the projection surface 150 and an adjusted projected image 500 corresponding to the gaze vector 164-12).
It would have been obvious to one of ordinary skill in the art prior to the filing date of the application to modify the projection system of Cui with the viewpoint adaptive projection system of Kursula in order for the user to view an image in the correct perspective (Kursula; para. 0017).
Regarding claims 8 and 17, Cui discloses wherein the instructions also cause the one or more processors are configured to: prior to projecting at least one of the visual information and additional visual information onto the different candidate surface information (para. 0010; identifying candidate projection areas and selecting a candidate projection area as a projection zone for projecting informational data from an autonomous aerial vehicle), determine that there are no obstructions within a projection path from the projector to the different candidate surface (para. 0037; candidate projection areas 151 and 152 are both free of obstructions, do not have rough surface features, and may be approximately the same distance from AAV 160 (and its projector, e.g., one of the modules 164)); and determine that a discernability of the at least one of the visual information and the additional visual information when projected onto the different candidate surface exceeds a threshold discernibility (para. 0054; there may be a maximum size beyond which the processing system will not attempt to identify contiguous areas as candidate projection zones. In one example, step 330 may identify those contiguous areas that are greater than the minimum size and that are relatively free of irregularities that would interfere with a visual projection of the information data. For instance, candidate projection areas may be completely free of irregularities, or may meet a requirement that any irregularity or irregularities are less than a threshold measure of a level of irregularity.).
Response to Arguments
Applicant's arguments filed 1/28/2026 have been fully considered but they are not persuasive.
Applicant argues on page 10 of the “Remarks” that Cui lack any explicit, per-surface "discernability" determinations based on a comparison of the surface's visual attributes to the specific attributes of the content to be projected, and they do not disclose threshold-based surface selection using that joint metric.
Examiner respectfully disagrees. Cui explicitly teaches when a candidate projection area is considered that the contiguous zones (para. 0036; AAV 160 may first identify areas that are relatively free of large objects or surface features that would interfere with a visual projection from AAV 160 and he search space 157 may include at least one member of the group of people 140, one of the non-group members 145, bushes 158, and a hydrant 159. These parts of the search space 157 may be deemed to have too many obstructions or non-smooth surface characteristics such that these parts of the search space 157 may be excluded from consideration as candidate projection areas) further, para. 0037; it can be seen in FIG. 1 that candidate projection areas 151 and 152 are both free of obstructions, do not have rough surface features, and may be approximately the same distance from AAV 160 (and its projector, e.g., one of the modules 164).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANELL L OWENS whose telephone number is (571)270-5365. The examiner can normally be reached 9:00am-5:00pm M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Minh-Toan Ton can be reached at 571-272-2303. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANELL L OWENS/Examiner, Art Unit 2882 14 April 2026
/BAO-LUAN Q LE/Primary Examiner, Art Unit 2882