DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Objections
Claim 14 is objected to because of the following informalities: Claim 14 recites “displaying a first visual effect that extends from an edge of the first media item outside of the first media item”. The underline portion seems incomplete. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 10-11, 15-16, 25 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US Patent No. 11810256 “Wu”) in view of Lee et al. (US Patent No. 11258965 “Lee”).
Regarding claim 1 Wu teaches A computer system (Fig. 11) configured to communicate with one or more display generation components and one or more input devices (Fig. 11 elements 1145 and 1135), the computer system comprising: one or more processors (Fig. 11 element 1110); and memory (Fig. 11 element 1130) storing one or more programs configured to be executed by the one or more processors (Col 2 lines 4-6 “In another example, a non-transitory computer-readable medium is provided that has stored thereon instructions that, when executed by one or more processors”), the one or more programs including instructions for:
receiving, via the one or more input devices, a sequence of one or more inputs corresponding to a request to change a display state of a first media item relative to a three-dimensional environment (ABSTRACT “The system can further determine a first portion of the 3D representation of the 2D input image corresponding to a target object in the 2D input image. The system can adjust a pose of the first portion of the 3D representation of the 2D input image corresponding to the target object. Col 9 lines 46-49 “The systems and techniques allow a user to provide user input to adjust the appearance (e.g., increase a size and/or change position) of an ROI or target object (e.g., selected by a user”);
and in response to receiving the sequence of one or more inputs, displaying, via the one or more display generation components, a first media item, in a respective display state (Col 26 lines 25-29 “At block 810, the process 800 includes generating a 2D output image having a modified version of the target object based on the adjusted pose of the first portion of the 3D representation of the 2D input image corresponding to the target object to be output on a display”), including:
in accordance with a determination that a camera that captured the first media item had a first field of view with a first value for a respective size parameter when capturing the first media item, displaying the first media item at a first size (Col 5 lines 40-45 “When capturing an image of a scene, photographers are often required to make a choice between capturing a wide field of view (FOV) that includes background objects and information that give context to a photograph and capturing a narrow field of view (e.g., with a telephoto lens) to capture foreground objects more prominently and/or with greater detail. Col 5 lines 54-57” In FIG. 1B, the image 110 is captured with a narrow FOV, and the person 112 near the camera (e.g., in the foreground) as well as the performer 114 are included prominently in the image 110 while only a limited portion of the background 116 is included in image 110”. Here height of the image 110 is smaller and part of person 112 is visible);
and in accordance with a determination that a camera that captured the first media item had a second field of view with a second value for the respective size parameter different from the first value for the respective size parameter when capturing the first media item, displaying the first media item at a second size, wherein the second size is different from the first size (Col 5 lines 40-45 “When capturing an image of a scene, photographers are often required to make a choice between capturing a wide field of view (FOV) that includes background objects and information that give context to a photograph and capturing a narrow field of view (e.g., with a telephoto lens) to capture foreground objects more prominently and/or with greater detail. Col 5 lines 58-59” In FIG. 1C, the image 120 is captured with a wide FOV, and the person 112, due to being near to the camera continues to be prominently displayed”. Here height of the image 120 is larger compare to image 110 and more part of person 112 is visible compare image 110 in Fig. 1B. So size of 110 is different than size of 120);
However Wu is silent about displaying a first media item, in a respective display state in the three-dimensional environment, including: displaying the first media item at a first size in the three-dimensional environment; displaying the first media item at a second size in the three-dimensional environment;
Lee teaches displaying a first media item, in a respective display state in the three-dimensional environment, including: displaying the first media item at a first size in the three-dimensional environment; displaying the first media item at a second size in the three-dimensional environment (Col 14 lines 2119-35 “ Referring to FIG. 12, a plane 1210 and a plane 1220 may be placed in a virtual 3D space (e.g., a Z axis) to correspond to depth values thereof…… In operation 1130, the electronic device may display at least a portion of the object on an image, based on the changed depth value of the object and the changed size of the object. Col 15 lines 18-24 “Referring to FIG. 14, in operation 1410, the electronic device may determine the FOV of the image. In operation 1420, the electronic device may determine the size increment or the size decrement of the object, based on changed depth values and FOVs of images. For example, referring to FIG. 15, an FOV of an image in which an object 1522 is composed may be greater than an FOV of an image in which an object 1524 is composed.”);
Wu and Lee are analogous art as they are from the field of image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Wu by displaying a first media item, in a respective display state in the three-dimensional environment, including: displaying the first media item at a first size in the three-dimensional environment; displaying the first media item at a second size in the three-dimensional environment as taught by Lee.
The motivation for the above is to provide the media item for better view.
Claim 25 is directed to “A non-transitory computer-readable storage medium” (Wu (Col 2 lines 4-6 “In another example, a non-transitory computer-readable medium is provided that has stored thereon instructions that, when executed by one or more processors”) and its elements are similar in scope and functions performed by the elements of claim 1 and therefore claim 25 is also rejected with the same rationale as specified in the rejection of claim 1.
Claim 26 is directed to a method claim and its steps are similar in scope and functions performed by the elements of claim 1 and therefore claim 26 is also rejected with the same rationale as specified in the rejection of claim 1.
Regarding claim 2 Wu modified by Lee teaches wherein: the first field of view with the first value for the respective size parameter is a narrower field of view than the second field of view with the second value for the respective size parameter; and the first size is smaller than the second size (Wu Col 5 lines 40-45 “When capturing an image of a scene, photographers are often required to make a choice between capturing a wide field of view (FOV) that includes background objects and information that give context to a photograph and capturing a narrow field of view (e.g., with a telephoto lens) to capture foreground objects more prominently and/or with greater detail. Col 5 lines 54-57” In FIG. 1B, the image 110 is captured with a narrow FOV, and the person 112 near the camera (e.g., in the foreground) as well as the performer 114 are included prominently in the image 110 while only a limited portion of the background 116 is included in image 110”. Here height of the image 110 is smaller and part of person 112 is visible).
Regarding claim 3 Wu modified by Lee teaches wherein: the first field of view with the first value for the respective size parameter is a wider field of view than the second field of view with the second value for the respective size parameter; and the first size is larger than the second size (Wu Col 5 lines 40-45 “When capturing an image of a scene, photographers are often required to make a choice between capturing a wide field of view (FOV) that includes background objects and information that give context to a photograph and capturing a narrow field of view (e.g., with a telephoto lens) to capture foreground objects more prominently and/or with greater detail. Col 5 lines 58-59” In FIG. 1C, the image 120 is captured with a wide FOV, and the person 112, due to being near to the camera continues to be prominently displayed”. Here height of the image 120 is larger compare to image 110 and more part of person 112 is visible compare image 110 in Fig. 1B. So size of 110 is different than size of 120).
Regarding claim 4 Wu modified by Lee teaches wherein: displaying the first media item at the first size in the three-dimensional environment includes displaying the first media item in the respective display state in the three-dimensional environment at a first distance from a viewpoint of a user of the computer system; and the first size is determined based on the first value for the respective size parameter and the first distance (Wu displays first media item at a first distance from viewpoint and Lee displays in 3d environment.
Wu Col 6 lines 56-60 “In some cases, if the distance of the target object 243 is at a known distance d from the center of projection 245, the point 246 (e.g., the 3D vertex) corresponding to the example pixel 242 can be located at a distance d from the camera center of projection 245 in the direction of ray r”
In Fig. 2C of Wu the size of the image 260 depends on the distance from viewpoint 245 as image became bigger compare to Fig. 2B ).
Regarding claim 5 Wu modified by Lee teaches wherein: displaying the first media item in the respective display state in the three-dimensional environment comprises displaying the first media item in the respective display state in the three-dimensional environment at a first distance from a viewpoint of a user of the computer system (Wu displays first media item at a first distance from viewpoint and Lee displays in 3d environment.
Wu Col 6 lines 56-60 “In some cases, if the distance of the target object 243 is at a known distance d from the center of projection 245, the point 246 (e.g., the 3D vertex) corresponding to the example pixel 242 can be located at a distance d from the camera center of projection 245 in the direction of ray r”).
Regarding claim10 Wu modified by Lee teaches wherein the first media item is a landscape photograph (Wu Fig. 1A shows landscape photograph).
Regarding claim11 Wu modified by Lee teaches wherein the three-dimensional environment is part of an extended reality environment (Wu Col 4 lines 21-26 “In some aspects, one or more of the apparatuses described above is, is part of, or includes a mobile device (e.g., a mobile telephone or so-called “smart phone” or other mobile device), a wearable device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device)”).
Regarding claim15 Wu modified by Lee teaches the one or more programs further including instructions for: receiving, via one or more input devices, a second sequence of one or more inputs corresponding to a request to change a display state of a respective media item relative to the three-dimensional environment; and in response to receiving the second sequence of one or more inputs: in accordance with a determination that the respective media item is a media item of a first type, displaying, via the one or more display generation components, the respective media item in a first display state in the three-dimensional environment; and in accordance with a determination that the respective media item is a media item of a second type different from the first type, displaying, via the one or more display generation components, the respective media item in a second display state in the three-dimensional environment, wherein the second display state is different from the first display state (Wu Col 5 lines 40-45 “When capturing an image of a scene, photographers are often required to make a choice between capturing a wide field of view (FOV) that includes background objects and information that give context to a photograph and capturing a narrow field of view (e.g., with a telephoto lens) to capture foreground objects more prominently and/or with greater detail”.
Here ”first type” refers to a wide FOV image which is a panoramic image and second type refers to narrow FOV image that is not panoramic).
Regarding claim16 Wu modified by Lee teaches wherein: the first type of media item is panoramic media item; and the second type of media item is a non-panoramic media item (Wu Col 5 lines 40-45 “When capturing an image of a scene, photographers are often required to make a choice between capturing a wide field of view (FOV) that includes background objects and information that give context to a photograph and capturing a narrow field of view (e.g., with a telephoto lens) to capture foreground objects more prominently and/or with greater detail”.
Here ”first type” refers to a wide FOV image which is a panoramic image and second type refers to narrow FOV image that is not panoramic).
Claim(s) 6, 12-13 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Wu modified by Lee and further in view of Rocket et al. (US Pat. Pub. No. 20220101593 “Rocket”).
Regarding claim 6 Wu modified by Lee is silent about wherein the first distance is selected to be greater than a threshold distance.
Rocket teaches first distance is selected to be greater than a threshold distance (“[0138]… In some embodiments, the adjusted position is determined based on a second type of correspondence that is different from the first type of correspondence and ensures that the adjusted position remains more than the threshold distance from the position of the viewpoint of the currently displayed view of the three-dimensional environment shown via the first display generation component (e.g., view 7304-a″, subsequent views shown via the first display generation component 7100, etc.)”);
Wu modified by Lee and Rocket are analogous as they are from the field of rendering of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee to have first distance is selected to be greater than a threshold distance as taught by Rocket.
The motivation for the above is to have the image seen by the user with an optimized size.
Regarding claim 12 Wu modified by Lee is silent about while receiving the sequence of one or more user inputs corresponding to the request to change the display state of the first media item relative to the three-dimensional environment, the three-dimensional environment is displayed with a first set of visual characteristics; and the one or more programs further include instructions for: in response to receiving the sequence of one or more inputs, reducing a visual prominence of the three-dimensional environment in which the first media item is displayed by modifying at least some visual characteristics of the first set of visual characteristics.
Rocket teaches while receiving the sequence of one or more user inputs corresponding to the request to change the display state of the first media item relative to the three-dimensional environment, the three-dimensional environment is displayed with a first set of visual characteristics; and the one or more programs further include instructions for: in response to receiving the sequence of one or more inputs, reducing a visual prominence of the three-dimensional environment in which the first media item is displayed by modifying at least some visual characteristics of the first set of visual characteristics (“[0220] In some embodiments, in accordance with a determination that the current location of the user corresponds to a location associated with a respective type of exercise, the computer system displays a view of the three-dimensional environment that corresponds to the respective type of exercise, including gradually reducing the visual prominence of the representation of the physical environment in the currently displayed view of the three-dimensional environment, while increasing visual prominence of virtual content corresponding to the respective type of exercise associated with the current location in the view of the three-dimensional environment”);
Wu modified by Lee and Rocket are analogous as they are from the field of rendering of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee by having while receiving the sequence of one or more user inputs corresponding to the request to change the display state of the first media item relative to the three-dimensional environment, the three-dimensional environment is displayed with a first set of visual characteristics; and the one or more programs further include instructions for: in response to receiving the sequence of one or more inputs, reducing a visual prominence of the three-dimensional environment in which the first media item is displayed by modifying at least some visual characteristics of the first set of visual characteristics as taught by Rocket.
The motivation for the above is to make media item as a center of attraction.
Regarding claim 13 Wu modified by Lee and Rocket teaches wherein reducing the visual prominence of the three-dimensional environment comprises: for a first region of the three-dimensional environment that immediately surrounds the first media item, modifying at least some visual characteristics of the first set of visual characteristics by a first amount; and for a second region of the three-dimensional environment that is different from the first region, modifying at least some visual characteristics of the first set of visual characteristics by a second amount different from the first amount, wherein the first amount is greater than the second amount (Rocket “[0398] Gradually reducing the second representation of the second portion of the physical environment, and gradually increasing prominence of virtual content corresponding to the first type of exercise in regions of the second view of the three-dimensional environment in which the second representation of the second portion of the physical environment has been gradually reduced”).
Regarding claim 23 Wu modified by Lee is silent about in response to receiving the sequence of one or more inputs, outputting first non-visual feedback in conjunction with displaying the first media item, in the respective display state in the three-dimensional environment;
Rocket teaches in response to receiving the sequence of one or more inputs, outputting first non-visual feedback in conjunction with displaying the first media item, in the respective display state in the three-dimensional environment (“[0106] In some embodiments, the display generation component displays different views of the three-dimensional environment in accordance with user inputs or movements that changes the virtual position of the viewpoint of the currently displayed view of the three-dimensional environment relative to the three-dimensional environment. [0353] Changing an audio output mode from a first audio output mode to a second audio output mode that has more computationally controlled variables that the first audio output mode, in accordance with a determination that the biometric data corresponding to the first user meets the first criteria, provides improved audio feedback to the user (e.g., improved audio feedback that the computer system has transitioned from the first level of immersion to the second level of immersion, improved audio feedback that the biometric data corresponding to the first user has met the first criteria, etc.)”);
Wu modified by Lee and Rocket are analogous as they are from the field of rendering of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee by having in response to receiving the sequence of one or more inputs, outputting first non-visual feedback in conjunction with displaying the first media item, in the respective display state in the three-dimensional environment as taught by Rocket.
The motivation for the above is to enhance the applicability of Wu by having enhanced feedback (Rocket [353]).
Wu modified by Lee and Rocket teaches while displaying the first media item in the respective display state in the three-dimensional environment, receiving, via the one or more input devices, a subsequent sequence of one or more inputs corresponding to a request to change the display state of the first media item from the respective display state to a second respective display state different from the respective display state; and in response to receiving the subsequent sequence of one or more inputs: displaying the first media item in the second respective display state; and outputting second non-visual feedback different from the first non-visual feedback in conjunction with displaying the first media item in the second respective display state (Rocket “[0353] Changing an audio output mode from a first audio output mode to a second audio output mode that has more computationally controlled variables that the first audio output mode, in accordance with a determination that the biometric data corresponding to the first user meets the first criteria, provides improved audio feedback to the user (e.g., improved audio feedback that the computer system has transitioned from the first level of immersion to the second level of immersion, improved audio feedback that the biometric data corresponding to the first user has met the first criteria, etc.)”).
Claim(s) 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Wu modified by Lee as applied to claim 1 above, and further in view of Kim et al. (US Pat. Pub. No. 20190005709 “Kim”).
Regarding claim 7 Wu modified by Lee is silent about wherein displaying the first media item in the respective display state in the three-dimensional environment includes displaying the first media item as though it is being projected onto a simulated curved surface.
Kim teaches displaying first media item in respective display state in three-dimensional environment includes displaying the first media item as though it is being projected onto a simulated curved surface (Kim “[0054] Embodiments of the present disclosure provide techniques for reducing effects of image content discontinuities. FIG. 9 illustrates operations of a first embodiment, in which an image rendering device may transform image content by projecting content from the different views of an image from a native domain of the image to a spherical projection. FIG. 9 illustrates application to the use case of FIGS. 7 and 8. In this embodiment, image content from the planar views 720, 730 may be transformed to a spherical projection 910. In this embodiment, the image rendering device may transform lengths of the object L1, L2 in the planar views 720, 730 to angular projections α1, α2 in the spherical projection 910; although FIG. 9 illustrates a two-dimensional of the concept, the operation may be performed on a 3D projection 910. Thereafter, all or a portion of the image content from the spherical projection 910 may be selected for rendering”).
Wu modified by Lee and Kim are analogous as they are from the field of rendering of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee to have included displaying the first media item in the respective display state in the three-dimensional environment includes displaying the first media item as though it is being projected onto a simulated curved surface as taught by Kim.
The motivation for the modification is to reduce effects of image content discontinuities (Kim [0054])
Regarding claim 8 Wu modified by Lee and Kim teaches wherein the first media item curves based on a curvature of the simulated curved surface and a length of the first media item (Kim “[0054] Embodiments of the present disclosure provide techniques for reducing effects of image content discontinuities. FIG. 9 illustrates operations of a first embodiment, in which an image rendering device may transform image content by projecting content from the different views of an image from a native domain of the image to a spherical projection. FIG. 9 illustrates application to the use case of FIGS. 7 and 8. In this embodiment, image content from the planar views 720, 730 may be transformed to a spherical projection 910. In this embodiment, the image rendering device may transform lengths of the object L1, L2 in the planar views 720, 730 to angular projections α1, α2 in the spherical projection 910; although FIG. 9 illustrates a two-dimensional of the concept, the operation may be performed on a 3D projection 910. Thereafter, all or a portion of the image content from the spherical projection 910 may be selected for rendering”).
Regarding claim 9 Wu modified by Lee is silent about wherein the first media item is a panoramic image.
Kim teaches first media item is a panoramic image (“[0091] Similarly, for the image format 1900 illustrated in FIG. 19, a portion of the panoramic view 1920 border the top view 1912 and a different portion of the panoramic view 1920 borders the bottom view 1914. These portions may be used to develop padding content for the top view 1912 and the bottom view 1914. Similarly, edge portions of the top and bottom views 1912, 1914 may be used to develop padding content for filtering the panorama view 1920”);
Wu modified by Lee and Kim are analogous as they are from the field of rendering of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee to have first media item as a panoramic image as taught by Kim.
The motivation for the above is to enhance the applicability of Wu by supporting different types of images.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Wu modified by Lee as applied to claim 1 above, and further in view of Abou-Hallawa et al. (US Pat. Pub. No. 20090199087 “Abou-Hallawa”).
Regarding claim 14 Wu modified by Lee is silent about wherein displaying the first media item in the respective display state in the three-dimensional environment comprises displaying a first visual effect that extends from an edge of the first media item outside of the first media item.
Abou-Hallawa teaches displaying a first visual effect that extends from an edge of item outside of item (“[0032] FIG. 2B shows the run of text 200 with a rich visual effect 202B applied thereto. In this example, the rich visual effect comprises a glow effect that provides a soft border that extends outside all edges of the run of text 200. Application of a glow effect as shown in FIG. 2B increases both the height and width of the run of text 200”);
Wu modified by Lee and Abou-Hallawa are analogous as they are from the field of rendering of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee by displaying a first visual effect that extends from an edge of the first media item outside of the first media item similar to displaying a first visual effect that extends from an edge of item outside of item as taught by Abou-Hallawa.
The motivation for the above is to get user’s attention by providing visual effects.
Claim(s) 17 is rejected under 35 U.S.C. 103 as being unpatentable over Wu modified by Lee as applied to claim 15 above, and further in view of Gigushinski et al. (US Pat. Pub. No. 20230283758 “Gigushinski”).
Regarding claim 17 Even though Wu modified by Lee teaches the second type of media item is a panoramic media item as shown above but is silent about the first type of media item is a spatial media item;
Gigushinski teaches the first type of media item is a spatial media item (“[0058]…..In step 516, the 1.sup.st and 2.sup.nd Tele cameras acquire stereo images (i.e. images with some or significant overlap between native FOV.sub.T1 and native FOV.sub.T2) of OOI 402 with high resolution”).
Wu modified by Lee and Gigushinski are analogous as they are from the field image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee by having the first type of media item as a spatial media item as taught by Gigushinski.
The motivation for the above is to enhance applicability of Wu by supporting different types of images.
Claim(s) 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over Wu modified by Lee as applied to claim 15 above, and further in view Fukushi (US Patent No. 10360658 “Fukushi”).
Regarding claim 18 Wu modified by Lee is silent about prior to displaying the first media item in the respective display state in the three-dimensional environment, displaying, via the one or more display generation components, the first media item in a collection display state different from the respective display state;
Fukushi teaches prior to displaying first media item in a respective display state in an environment, displaying, via one or more display generation components, the first media item in a collection display state different from the respective display state (Fig 4(b) shows thumbnails display prior to Fig. 4(a) where it shows viewpoint image C.
Col 4 lines 44-50 “FIG. 4 (a) to (c) are diagrams illustrating an example of changing the display image that is displayed on the display device 20 of the first embodiment. The omnidirectional image 30 in FIG. 4 (b) is a case in which the partial omnidirectional image that is an image of the region including the viewpoint D is included in the center, and indicates the display image before the change”);
Wu modified by Lee and Fukushi are analogous as they are from the field image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee by having prior to displaying the first media item in the respective display state in the three-dimensional environment, displaying, via the one or more display generation components, the first media item in a collection display state different from the respective display state similar to having prior to displaying first media item in a respective display state in an environment, displaying, via one or more display generation components, the first media item in a collection display state different from the respective display state as taught by Fukushi.
The motivation for the above is to provide user controllability for image selection.
Wu modified by Lee and Fukushi teaches wherein: in the respective display state, the first media item is displayed at a first distance from a viewpoint of a user of the computer system; and in the collection display state, the first media item is displayed at a second distance from the viewpoint of the user of the computer system, wherein the second distance is closer to the viewpoint of the user of the computer system than the first distance (Fukushi Col 4 lines 44-50 “FIG. 4 (a) to (c) are diagrams illustrating an example of changing the display image that is displayed on the display device 20 of the first embodiment. The omnidirectional image 30 in FIG. 4 (b) is a case in which the partial omnidirectional image that is an image of the region including the viewpoint D is included in the center, and indicates the display image before the change”.
The thumbnail images in Fig 4(a) has a closer distance from user than the expanded image 32).
Regarding claim 19 Wu modified by Lee and Fukushi teaches wherein, in the collection display state, the first media item is displayed at a third size in the three-dimensional environment, and the third size is independent of a field of view of a camera that captured the first media item (Fukushi Fig 3 shows thumbnail images in 40 where thumbnail images has a different size and independent of fov).
Regarding claim 20 Wu modified by Lee and Fukushi teaches the one or more programs further including instructions for: while displaying the first media item in the collection display state, receiving, via the one or more input devices, a first navigation input that includes movement in a first direction; and in response to receiving the first navigation input, navigating from the first media item to a second media item, wherein: the first media item is a first type of media item; and the second media item is a second type of media item different from the first type of media item (Fukushi Col 6 lines 4-10 “Moreover, operation made to the viewpoint listing image is operation to choose a thumbnail of a partial omnidirectional image on the viewpoint listing image, operation to change a thumbnail of a partial omnidirectional image to be displayed at the predetermined position in the viewpoint listing image by scrolling the viewpoint listing image, and the like”.
In Fukushi Fig. 4(b) First media item shows a fireplace with viewpoint D and Fig. 4(c) shows second media item with walls and window with viewpoint E, which is different than first media item).
Regarding claim 21 Wu modified by Lee and Fukushi teaches the one or more programs further including instructions for: while displaying the first media item in the collection display state, receiving, via the one or more input devices, a first selection input; and in response to receiving the selection input, expanding the first media item (Fukushi from Fig. 4(b) to Fig. 4 (a) shows expanded view of viewpoint image C.
Col 6 lines 51-53 “ FIG. 4 (c) indicates a case in which the accepting unit 15 has accepted an instruction to change to the viewpoint E from a user”).
Claim(s) 22 is rejected under 35 U.S.C. 103 as being unpatentable over Wu modified by Lee as applied to claim 1 above, and further in view of Chen et al. (US Pat. Pub. No. 20170302719 “Chen”).
Regarding claim 22 Wu modified by Lee teaches receiving, via the one or more input devices, a third sequence of one or more inputs corresponding to a request to change a display state of a first respective media item relative to the three-dimensional environment; and in response to receiving the third sequence of one or more inputs, displaying, via the one or more display generation components, the first respective media item in the respective display state in the three-dimensional environment, including displaying the first respective media item at a first respective size, wherein: the first respective media item was captured by a first camera that had a first respective field of view with a first respective value for a first respective size parameter when the first respective media item was captured (Wu Col 10 lines 36-42 “A user can provide a user input (e.g., via a touchscreen interface, using a voice input, using a gesture input, a gaze input, etc.) to indicate or select one or more ROIs (which can be associated with a target object or multiple target objects from the detected one or more objects) in the color image for which the user wants to adjust (e.g., make larger, change rotation, etc.). (Col 5 lines 40-45 “When capturing an image of a scene, photographers are often required to make a choice between capturing a wide field of view (FOV) that includes background objects and information that give context to a photograph and capturing a narrow field of view (e.g., with a telephoto lens) to capture foreground objects more prominently and/or with greater detail. Col 5 lines 58-59” In FIG. 1C, the image 120 is captured with a wide FOV, and the person 112, due to being near to the camera continues to be prominently displayed”) but is silent about the first respective media item was cropped by a first crop amount after being captured by the first camera; and the first respective size is determined based on the first respective value and the first crop amount.
Chen teaches respective media item was cropped by a first crop amount after being captured by the first camera; and the first respective size is determined based on the first respective value and the first crop amount (“[0079]……. The zoom controller may implement one or more of the aspects of the improved zooming strategy described above. For example, the zoom controller may control the zoom region in order to follow an object of interest and/or stabilize overall scene movement. Additionally or alternatively, the zoom controller can smoothly update the FOV when the size of the tracking region changes (to keep the object of interest at a reasonable size ratio to the zoom region). Additionally or alternatively, the zoom controller may smoothly return to the original field of view when the tracked object is missing in an image (e.g., frame) and/or when tracking is lost. The zoom controller can produce cropped images (e.g., cropped frames). For example, the zoom controller may crop and/or scale (and/or control another component to crop and/or scale) the input images according to the zoom region. For instance, the zoom controller may crop and/or discard any image content that is outside of the zoom region. In some configurations, the zoom controller may scale the cropped images to a particular size (e.g., the size of a display, the size of a window on a display, etc.). For example, an upscaling process may be performed to upscale the cropped images to the size of a display or a window on the display”);
Wu modified by Lee and Chen are analogous as they are from the field image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee by having respective media item was cropped by a first crop amount after being captured by the first camera; and the first respective size is determined based on the first respective value and the first crop amount as taught by Chen.
The motivation for the above is to control the size of the item for optimum display.
Claim(s) 24 is rejected under 35 U.S.C. 103 as being unpatentable over Wu modified by Lee as applied to claim 1 above, and further in view of Fukushi and Rocket.
Regarding claim 24 Wu modified by Lee is silent about displaying, via the one or more display generation components, a representation of the first media item in a first media collection user interface; while displaying the representation of the first media item in the first media collection user interface, receiving, via the one or more input devices, a first user input corresponding to a user request to display the first media item in a first expanded view; in response to receiving the first user input: displaying, via the one or more display generation components, the first media item in a first expanded view that is different from the first media collection user interface;
Fukushi teaches displaying, via the one or more display generation components, a representation of the first media item in a first media collection user interface; while displaying the representation of the first media item in the first media collection user interface, receiving, via the one or more input devices, a first user input corresponding to a user request to display the first media item in a first expanded view; in response to receiving the first user input: displaying, via the one or more display generation components, the first media item in a first expanded view that is different from the first media collection user interface (Fukushi Col 6 lines 4-10 “Moreover, operation made to the viewpoint listing image is operation to choose a thumbnail of a partial omnidirectional image on the viewpoint listing image, operation to change a thumbnail of a partial omnidirectional image to be displayed at the predetermined position in the viewpoint listing image by scrolling the viewpoint listing image, and the like”.
Wu modified by Lee and Fukushi are analogous as they are from the field image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee by having displaying, via the one or more display generation components, a representation of the first media item in a first media collection user interface; while displaying the representation of the first media item in the first media collection user interface, receiving, via the one or more input devices, a first user input corresponding to a user request to display the first media item in a first expanded view; in response to receiving the first user input: displaying, via the one or more display generation components, the first media item in a first expanded view that is different from the first media collection user interface as taught by Fukushi.
The motivation for the above is to provide user controllability for image selection.
Wu modified by Lee and Fukushi is silent about outputting third non-visual feedback in conjunction with displaying the first media item in the first expanded view;
Rocket teaches outputting third non-visual feedback in conjunction with displaying a first media item in a first expanded view (“[0106] In some embodiments, the display generation component displays different views of the three-dimensional environment in accordance with user inputs or movements that changes the virtual position of the viewpoint of the currently displayed view of the three-dimensional environment relative to the three-dimensional environment. [0353] Changing an audio output mode from a first audio output mode to a second audio output mode that has more computationally controlled variables that the first audio output mode, in accordance with a determination that the biometric data corresponding to the first user meets the first criteria, provides improved audio feedback to the user (e.g., improved audio feedback that the computer system has transitioned from the first level of immersion to the second level of immersion, improved audio feedback that the biometric data corresponding to the first user has met the first criteria, etc.)”);
Wu modified by Lee and Fukushi and Rocket are analogous as they are from the field of rendering of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu modified by Lee and Fukushi by outputting third non-visual feedback in conjunction with displaying a first media item in a first expanded view as taught by Rocket.
The motivation for the above is to enhance the applicability of Wu by having enhanced feedback (Rocket [353]).
Wu modified by Lee and Fukushi and Rocket teaches while displaying the first media item in the first expanded view, receiving, via the one or more input devices, a second user input corresponding to a user request to display the first media item in the respective display state in the three-dimensional environment; and in response to receiving the second user input: displaying, via the one or more display generation components, the first media item in the respective display state in the three-dimensional environment; and outputting fourth non-visual feedback that is different from the third non-visual feedback in conjunction with displaying the first media item in the respective display state (Rocket [0353] Changing an audio output mode from a first audio output mode to a second audio output mode that has more computationally controlled variables that the first audio output mode, in accordance with a determination that the biometric data corresponding to the first user meets the first criteria, provides improved audio feedback to the user (e.g., improved audio feedback that the computer system has transitioned from the first level of immersion to the second level of immersion, improved audio feedback that the biometric data corresponding to the first user has met the first criteria, etc.)”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAPTARSHI MAZUMDER whose telephone number is (571)270-3454. The examiner can normally be reached 8 am-4 pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at (571)272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAPTARSHI MAZUMDER/ Primary Examiner, Art Unit 2612